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PREFACE 



In a society where employment opportunities for many groups of 
disadvantaged people are limited, the mentally retarded often encounter 
particular difficulty in achieving steady work in the regular labor market. 
This study, the first of two related research reports, describes the operat- 
ing experience of the Struci red Training and Employment Transitional 
Services (STETS) demonstration, a program designed to prepare mentally 
retarded young adults for regular private sector employment. 

In considering the lessons of this study, one must first recognize the 
fact that transitional employment initiatives such as STETS represent a 
major change over a relatively short time span. Few of us working in the 
field of mental retardation in the 1960s would have predicted that 
systematic efforts would soon be made to bring mentally retarded citizens 
into the regular labor market. The process, which began with a few small 
pilot projects in the 1970s, coincided with a broader trend toward M main- 
streaming" disabled citizens. The STETS demonstration has built on these 
encouraging earlier experiences by conducting a somewhat larger program 
within the framework of a comprehensive research design. The research is 
capable of providing hard evidence in a field too often characterized by 
anecdote and impression. 

Nevertheless, the progress to date should not obscure the many 
challenges facing programs that seek to help mentally retarded persons find 
and keep work. While this study points to a number of important achieve- 
ments, it also highlights areas where program operators can expect to 
confront difficulties. Managing an employment program for mentally 



retarded workers requires effective recruitment strategies (directed at 
potential participants, their parents and the agencies that serve them), a 
sensitive but firm hand in acclimating participants to the dictates of the 
workplace, and above all, the ability to develop sufficient, appropriate 
jobs for participants. The lessons described in this report on the five 
sites taking part in the STETS demonstration provide a useful guide to 
others — program operators, public officials and the business community — 
who become involved in similar initiatives in the future. 

The next report on the STETS demonstration, due in mid-1985 , will 
determine whether the particular mix of jobs and services provided by the 
program improved the employment, earnings and independence of participants 
after they had left the program. This study, based largely on interviews 
with individuals randomLy assigned to either a participant or control 
group, will also compare the economic benefits and costs of the program. 
In addition to providing a reliable basis for further development of public 
policies in this area, the evaluation will yield important lessons in the 
application of social science research tools to employment programs for 
mentally retarded citizens. 

The studies will not, of course, provide all the answers. The process 
of learning more about alternative approaches — and how best to integrate 
them into the mainstream of the service delivery system for this population 
— will need to continue. 

Barbara B. Blum 
President 
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EXECUTIVE SUMMARY 



The subject of this report is the implementation of a transitional 
employment strategy designed to help young, mentally retarded adults become 
competitively employed. The Structured Training and Employment Transi- 
tional Services (STETS) program was operated from the fall of 1981 through 
December 1983, with follow-up services for a small number of participants 
continuing into early 1984, Principal funding was provided by the U.S. 
Department of Labor, with a planning grant from The Ford Foundation. Local 
projects also raised almost 60 percent of their operating revenues from a 
variety of public and private resources. 

The STETS program was managed by several different types of agencies 

in five cities: a sheltered workshop in Cincinnati; an "affirmative 

industry," which in many ways resembles a sheltered workshop, in St. Paul; 

a state agency in Tucson; and nonprofit training agencies in New York and 

Los Angeles. Each of these sites was part of a major research effort 

focusing on three sets of questions: 

Implementation : Is the program model feasible to implement, 
and what advantages and disadvantages are associated with 
alternative strategies? 

Impact: How effective is the program in increasing partici- 
pants' post-program employment, earnings and independence, and 
how does it affect their use of alternative services and the 
receipt of transfer payments? 

Benefits and Costs: How do the economic benefits of STETS 
compare to the costs of operating the program? 

To answer impact and benefit-cost questions, 496 applicants to STETS 

were randomly assigned to either the program (the experimental group) or to 
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a control group. Members of the control group could not receive STETS 

m 

services but could seek other services available in the community. Data 
for the impact analysis were collected through in-person interviews with 
members of the experimental and control groups soon after random 
assignment, and again at 6, 15 and 22 months after the initial interviews. 

The Manpower Demonstration Research Corporation (MDRC), an organi- 
zation that designs and evaluates innovative social programs, has held 
overall responsibility for both the operation and evaluation of the STETS 
demonstration. MDRC was directly responsible for the implementation analy- 
sis, while a subcontractor, Mathematica Policy Research, Inc., i& con- 
ducting the impact and benefit-cost components of the research (scheduled 
for completion in 1985) under the general supervision of MDRC. 



I. The STETS Model 

STETS grew out of the Supported Work demonstration, a transitional 

employment program which proved effective in helping certain groups of 

hard-to-employ individuals enter the labor market. Supported Work 

emphasizes close supervision, peer support and a gradual increase in 

productivity demands. In STETS, these basic principles were incorporated 

into a three-stage model with the following features: 

Phase 1 : Participants were given up to 500 hours of paid employ- 
ment, along with training and support services, 
provided in a low-stress work environment. This intro- 
duction to work took place in sheltered settings in 
Cincinnati, Tucson and St. Paul, and in public and 
nonprofit agencies in New York and Los Angeles. In 
the two non-sheltered sites, wage subsidies were 
available to employers. 

At all sites, program counselors assessed, trained and 
monitored participants as they performed their jobs, 
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and provided both individual and group counseling on 
job-related issues. 

Phase 2 ; Participants vert placed into non-sheltered positions 
that required more responsibility than Phase 1 jobs, 
and counselors continued to provide training and sup- 
port. While jobs could be subsidised or unsubsidized , 
and in the public, nonprofit or private sectors, Phase 
2 was expected to operate as an on-the-job training 
experience. Thus, an emphasis was placed on developing 
positions that could evolve into regular jobs and ones 
that entailed the kinds of demands normally associated 
with such jobs. 

Participants were to stay no longer than 12 months in 
Phases 1 and 2 combined, within a 15-month period. They 
made the transition from Phase 2 to Phase 3 when both 
their employers and program staff viewed them as 
regular members of tue company's workforce. 

Phase 3 : All jobs at this stage were unsubsidized. Staff mem- 
bers offered participants support and guidance on an 
as-needed basis for an additional six months to aid in 
their adaptation to regular employment. Generally, the 
level of support was much lower than in earlier stages 
of the program. 

*;he STETS target population was an 18- to 24-year-old group who had 
full-scale IQ scores between 40 and 80 or other verifiable evidence of 
moderate to borderline retardation. Additionally, eligible persons could 
have only limited prior work experience: They could not have been employed 
in a full-time unsubsidized job for more than six consecutive months in the 
two years before program entry, and they could not be working in an 
unsubsidized job for more than ten hours per week at the time of 
enrol Iment. 



II, Lessons on Program Operations, 

It is important to note that STETS operated for a relatively brief 
period of time, much of it marked by funding uncertainty. Consequently, 



the sites -Ud not have an opportunity to reach a stable period of 
operations- 'everthelecs, much has been learned about the program. 

The most general conclusion of the implementation analysis is that the 
STETS transitional employment approach is feasible and can be operated on a 
relatively large scale (e.g., 40 to 50 slots) by different types of organi- 
zations using diverse methods. The sites have shown, with varying degrees 
of success, that they could recruit participants, develop training posi- 
tions in real work settings, provide trainees with helpful support 
services, and move many into unsubsidized jobs. The average length of 
enrollment was 10.4 months, an d 42 percent of all enrol lees were placed 
into regular jobs. The average cost of operating the program was $7,553 
per participant and $8,715 per year of service, with actual costs varying 
widely across the sites. 

STETS proved, however, to be a challenging program to ope.-ate. One 
general difficulty was the local funding situation, which sometimes con- 
strained the sites' flexibility in implementing the model. For example, 
some funding agencies strictly limited the circumstances under which they 
would provide wage subsidies for participants; others were interested in 
funding only one component of the program or one category of expenses. 
Future programs would therefore benefit from more flexible funding 
arrangements or multiple sources of revenue without overlapping 
restrictions. 

Other lessons on operating STETS are summarized below. 
A. Recruitmenc and Referral 

STETS recruitment was generally effective, resulting in a reasonably 
diverse group of participants, ones who were likely to encounter signi- 



ficant employment problems. Enrollees were 60 percent male and 40 percent 
female; over two-fifths were black or Hispanic, They were, on average, 20 
years old, and 41 percent had at least one secondary handicap in addition 
to mental retardation. One-third had never attended a mainstream school 
since the age of 14, and almost 90 percent had not taken any classes as 
part of a regular curriculum. The average IQ of enrollees was 64. Almost 
half were mildly retarded, and 38 percent were in the borderline range. 
The proportion of moderately retarded persons. 13 percent, was lower than 
expected. 

While several reasons could be offered for the sites' difficulty in re- 
cruiting moderately retarded persons — including the fact that they are 
much less prevalent than mildly and borderline retarded persons in the 
general population — the experience of the Cincinnati and Tucson sites 
suggests that linkages to other agencies serving this group are parti- 
cularly useful. Some of the sites lacked these contacts. 

B. Work Experience. Assessment and Training 

During their tenure in STETS« participants manifested a variety of 
work-related problems that staff attempted to correct or improve. Among 
the problem areas were attendance and punctuality; interaction with 
supervisors and co-workers ; concentration and endurance; initiating, 
completing and changing tasks; grooming and hygiene; work speed; 
remembering and understanding how to do certain tasks; and working 
according to a preset schedule. While not all participants had difficul- 
ties in each area, the problems were common enough to signal a real need 
for work experience, training and support services. This finding supports 
the program's goal of providing those interventions and gradually increas- 



ing the demands and responsibilities of the jobs for participants. 



A number of implementation lessons can be drawn from the analysis of 
the sites' experiences in this area: 

• Both the sheltered and non-sheltered approaches in Phase 1 are 
feasible t although each has advantages and disadvantages . The 
sheltered approach makes it easier to monitor and instruct 
participants, reduces the number of external jobs to be devel- 
oped, and possibly allows the program to be less selective in 
recruiting clients, since relationships with outside employers 
are not at risk. The non-sheltered approach, in contrast, 
provides participants wirh working environments and types of 
jobs (usually in the service area) which are similar to those 
they will face after Phase 1. The non-sheltered approach may 
also improve the Phase 1 performance of participants who 
resent working in sheltered workshops or are ready for more 
demanding jobs. 

• Observational tech niques are faore important than formal test- 

i-Pft — ill assessing participants' work-related problems and 

Meds. At best, formal testing, which was conducted at two 
sites, served as an adjunct to observation, but was not 
considered essential by staff members. 

• The guid elines restricting participants' length of stay in the 
program were feasible. The 12-month limit on active partici- 
pation in Phases 1 and 2 combined proved reasonable, since 
most participants entered unsubsidized jobs or were otherwise 
terminated from the program in a shorter amount of time. 
Furthermore, the 500-hour limit on Phase 1 activities appeared 
useful for keeping pressure* on staff members to develop Phase 
2 jobs in a timely manner and to move participants expedi- 
tiously into higher-stress and generally more realistic work 
settings. 

• Participants can be trained in real work settings during Phase 
2 without unduly burdening the employers . That their work was 
generally satisfactory is seen in a survey of employers who 
rated* the vast majority of Phase 2 participants ac performing 
as well as or better than other new workers in similar jobs in 
five areas of performance: attendance (85 percent), 
punctuality (89 percent), motivation (79 percent), and quality 
of work (78 percent). Fifty-four percent of participants 
received positive ratings on all five measures, while almost 
46 percent received negative ratings in one or more areas. 

• The maj ority of employers responding to 'the survey (86 per- 
cent) said they wo uld become involved again in the program if 
asked to do so. Further evidence of employer satisfaction's 
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the observation that 66 percent of participants who entered 
Phase 3 jobs were hired by their Phase 2 employers. 



• Public, nonprofit or for-profit organiaat ions can be used for 
Phase 2 training . Public and nonprofit agencies may be more 
•willing to allow program staff to monitor participants and to 
intervene at the worksite. However, for-profit firms gener- 
ally provide a greater opportunity for a Phase 2 job to turn 
into a permanent job. 

# While the level of program support can be reduced during Phase 
3. these services should not be abruptly eliminated . They 
continue to be important to many participants, who require at 
least some assistance in adjusting to regular employment. 

The question of who among the mentally retarded can handle the demands 
of STETS and benefit from the program is another important operational 
issue. Accordingly, this study examined the in-program performance of 
various subgroups of enrollees. The subgroups were defined by background 
characteristics at the time of enrollment, including age, sex, level of 
mental retardation, and past employment-related and educational experi- 
ences. While some differences were apparent, a substantial proportion of 
each subgroup analyzed earned positive evaluations from Phase 2 employers 
and went on to unsubsidized jobs. This evidence suggests that future 
programs ought to continue the STETS policy of targeting services to a 
fairly broad range of mentally retarded individuals. The impact analysis 
will study this issue further, using the control group and longer-term 
follow-up data to assess whether the program's impacts vary by subgroup. 

C. Job Development 

In attempting to assist a large number of clients, an employment 
program is largely dependent on the effectiveness of the job development 
effort. In STETS, job development proved to be difficult and time-concum- 
ing, often causing serious delays in the movement of clients through the 
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program. Problems often resulted from the sites' overall lack of 
experience in finding jobs for this population, compounded by the fact that 
job developers had only limited time in which to strengthen techniques and 
develop positive "track records" with employers. In addition, participants 
had characteristics other than mental retardation (such as their young age) 
that could increase the reluctance of employers to give them jobs. , Other 
problems were the deep economic recession during the demonstration period 
and, at certain >ites, strategies that proved unproductive, such as 
individual job development rather than the creation of job pools, and not 
assigning job development tasks to a specialist. 

Despite these constraints, many employers agreed to accept STETS 
workers. Moreover, employers agreed to pay the full wage for 37 percent of 
Phase 2 participants. This outcome exceeded the expectations of program 
planners and reduced operating costs. 

The employer survey showed that, among Phase 2 employers, 78 percent 
stated that altruism, expressed as a desire to help participants or the 
community, was a major reason for taking on STETS trainees. Other rele- 
vant factors, rated on a scale from "very important 11 to "not important at 
all," were: 

# Thrrty-four percent of the employers indicated that the 
financial^ subsidies during Phase 2 were very important in 
their decision to take on trainees. 

# Among for-profit employers, 19 percent said that the Targeted 
Jobs Tax Credit was very important in their decision. 

# Fifty-seven^ percent of all employers surveyed said that 
program assistance in training and monitoring participants was 
very important, and 38 percent said that participants' prior 
training, in Phase 1 was very important. 

These factor; would not, of course, ensure th^t participants would 
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keep their jobs or obtain Phase 3 positions with the same employer. For 
this group, as for non-disabled persons, work performance is the main 
determinant of job retention. 

Eighty percent of the Phase 3 jobs developed were in for-profit firms. 
Fifty-three percent were jobs in the service fields, and 19 percent 
clerical and sales positions. The average starting wage was $3.63, and 
half of the jobs provided over 30 hours of weekly work. 

While most Phase 3 jobs fell into a relatively narrow range (e.g., 
janitor, porter, dishwasher, food service worker, housekeeper, mailroom 
clerk, stock clerk, messenger), the positions were located in diverse 
settings. Thus, future programs of this type can seek jobs from a wide 
variety of businesses. However, positions in larger firms may tend to take 
longer to develop because of the many steps that precede hiring decisions. 
Preliminary Results from the Impact Analysis 
A preliminary analysis of the second wave of interviews, conducted 
with membt-s of the experimental and control groups as part of the impact 
analysis by Mathematica Policy Research, Inc., offers evidence from another 
perspective on the feasibility of the STETS approach. The analysis focuses 
on short-term, in-program impacts, and it shows that, six months after 
random assignment, over two-thirds of the experimental group was in paid 
training or non-training jobs, higher than the rate for the control group. 
Moreover, more experimentals than controls were involved in either jobs, 
training or school activities. Thus, the program appeared to achieve its 
short-term goal: increasing the employment-related services received by 
enrol lees. 

The final impact and benefit-cost report on STETS will more fully 
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assess the effectiveness of the program by comparing the longer-term labor 
market and other achievements of the experimental and control groups. It 
will also determine whether or not the economic benefits of STETS exceeded 
its costs. * 
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A TRANSITIONAL EMPLOYMENT STRATEGY 



FOR THE MENTALLY RETARDED 



THE FINAI. STETS IMPLEMENTATION REPORT 



CHAPTER 1 
INTRODUCTION 

The last few decades have witnessed fundamental changes in the rights 
aud expectations of disabled persons, and in the ways that society has 
responded to their particular circumstances. At the core of this transfor- 
mation is the ideology of normalization: a set of principles that would 
bring disabled persons into the mainstream of American life by giving them 
access to the social roles and opportunities available to their non-handi- 
capped peers. This ideology is reflected in such notable legislative 
achievements as Sections 503 and 504 of the Rehabilitation Act of 1973 (in 
which discrimination against handicapped persons is forbidden in anv 
federally assisted activity) and the Education for All Handicapped Children 
Act of 1975, which requires public schools to provide a "free and appro- 
priate education" to all 5- to 21-year-old handicapped youths in the least 

J 

restrictive environment consistent with their needs. 

Competitive employment — that is, regular unsubsidized employment in 
the competitive labor market — can be an important part of the process of 
normalization. Through employment, disabled persons can interact with non- 
handicapped persons, begin to acquire the financial means to become self- 
auff xclerrt7^an3^share in^the dignity that regular work can provide. As one 
author said of the mentally retarded (the category of disabled persons 
studied in this report), "The best way (they) can prove to themselves and 
others that they are 'normal' Is to have a job" (Ingalls, 1978: 409). 

This document is tfie final implementation report on a demonstration 
program that sought to help young, mentally retarded adults with poor or no 
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work histories make a relatively quick transition to competitive employ- 
ment. Known as STETS — Structured Training and Employment Transitional 
Services — the program was run as a research demonstration in five cities, 
with local operations beginning in late 1981 and ending (except for follow- 
up activities with a few participants) in December 1983. During ^his 
period, the program enrolled 284 young adults. National funding was 
provided by the United States Department of Labor (DOL) and The Ford Founda- 
tion, wh^ch supported the project's initial planning. In addition, almost 
60 percent of program operating costs in the five cities was raised locally 
from a variety of public and private sources. 

The Manpower Demonstration Research Corporation (MDRC), a nonprofit 
corporation formed in 1974 to develop, administer and evaluate innovative 
social programs, has overall management and research responsibility for the 
STETS demonstration. In its management role, MDRC (with the assistance of 
others in the field) developed the program model, selected local 
organizations to implement it, and monitored the local sites to ensure that 
they were operating within specified program guidelines. The extensive 
research plan is described in a subsequent section of this chapter. 

This report opens vith a brief discussion of the nature of the problem 
which STETS sought to address. 

!• Obstacles to Employment Among the Mentally Retarded * • 

It is widely accepted by researchers and policymakers that the employ- 
ment problems of persons considered to be mentally retarded are numerous, 
but that, because of a lack of convincing research, their exact scope is 
difficult to determine.* Some generalizations are nevertheless possible. 
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For example, the evidence suggests that many retarded persons do eventually 
become competitively employed for at least part of their adult lives, parti- 
cularly in their middle to late twenties (although little is known about 
the quality and stability of their jobs). This is especially true of the 
mildly retarded (commonly defined as persons having IQ ranges from 52 to 68 
on the Stanford-Binet test, or between 55 and 69 on the Wechsler Scale) and 
the borderline retarded (an IQ range from 70 to 85 on the Wechsler Scale). * 
Lack of employment is more extensive among the moderately retarded (IQs of 
36 to 51 on the Stanford-Binet and 40 to 54 on the Wechsler Scale) and, the 

severely retarded (20 to 35, and 25 to 39 on the two srcales). (See Conley, 

o 

1983; IngaHs, 1978.) 

The evidence also indicates, however, that even the least disabled of 
the mentally retarded groups have more difficulty finding and keeping jobs 
than do their non-retarded peers. Many need special assistance to .enter 
and remain in the competitive labor market, and even then, proportionally 
fewer Tiold regular jobs. Among those who do, earnings are lower. The 
employment obstacles faced by the mentally retarded (which often affect 
physically handicapped persons as well) in both obtaining and retaining 
jobs are -multiple: negative employer attitudes, their own deficiencies in 
work skills and habits, and government policies that may discourage 
disabled persons from seeking competitive employment. These barriers are 
briefly discussed below. 

Studies examining employers' hiring attitudes, while few in number, 

* * 
have typically found that the mentally retarded are among the least 

preferred of potential workers. In a. comprehensive national overview of 

the needs of severely disabled persons, the Urban Institute reported that : 



Virtually all the studies on employer attitudes have found 
that large proportions of employers disfavor hiring disabled 
people, . . .There are also indications that certain groups of 
the disabled) particularly those with mental and behavioral 
disabilities and those with severe or aggrava table physical 
disabilities, are more disfavored by employers than others, 
though the evidence on this question is not entirely 
consistent (1975: 324). 

When mentally retarded persons do overcome negative attitudes and find 

employment, their own w?rk limitations may make it difficult for them to 

retain the jobs. Research has found that, among the more substantially 

retarded persons, slow work performance and an inability to change routine 

are important reasons for a job loss.^ Inadequate work habits are also 

common, and are a major reason among more mildly retarded persons for loss 

of employment. All of these factors seem more important than an inability 

to perform the work. According to one review of the literature: 

Virtually all studies indicate that when retarded people fail 
on a job, it is not because they cannot perform the task... 
(Gold, 1975: 256). 

Similarly, another researcher states: 

There are a great many jobs in this society that require no 
intellectual abilities at all, jobs such as stuffing enve- 
lopes or routine assembly skills or certain factory jobs that 
are highly repetitive, and mildly retarded people are able to 
perform most of these as well as anyone else. The reason why 
mentally retarded people lose their jobs is almost always 
because of some personality or behavior problems: they cannot 
get along with co-workers or with their boss or they are late 
or absent too frequently or they are unable to concentrate 
for long periods of time or they have peculiar, mannerisms or 
a>e emotionally unstable (ingalls, 1978: 409; see also Con- 
vey, 1973: 336). - 

Two major public income support systems are available to the disabled. 

Supplemental Security Income, or SSI, is a federally-administered cash 

assistance program for blind, disabled or old persons with limited 

resources and income, and is authorized under Title XVI of the Social 



-A- 



Security Act. Social Security Disability Insurance benefits (SSDI), 
authorized under Title II of that same law, are federal disability benefits 
paid to workers who have contributed to the Social Security trust funds. 
However, since benefits are reduced or eliminated when recipients gain 
income from jobs, it is possible for both programs to adversely affect the 
employment incentives for retarded persons. The extent of the disincentive 
is not known, but r for example, a disabled person earning more than $300 
per month in 1980 would have been judged capable of "substantial gainful 
activity" (according to Social Security Administration guidelii^-s) and 
would no longer have been considered eligible for SSI payments. 5 Medicaid 
benefits, whi^ch are provided to SSI recipients, would have also been 
forfeited in most circumstances. 6 

Thus, many retarded persons receiving benefits may be discouraged from 
seeking or keeping jobs because of uncertainty about whethex they will 
succeed in their jobs. 7 Fear of financial loss is also compounded by the 
complexity of the rules governing the public benefits programs. 

II • Employment Programs for Retarded Persons 

While the obstacles to competitive employment are considerable, many 
retarded' persons do aspire to work, and a variety of programs attempt to 
help them achieve that goal. 

Traditionally, nonprofit sheltered workshops have been a major source 
of job training and employment (as well as other rehabilitative services) 
for the disabled. According to one study t nearly 3,000 sheltered workshops 
in 1976 served more than 145,000 clients on a daily basis, 61 percent of 
them mentally retarded (U.S. Department of Labor, 1979)-— The study esti- 



mated that, during that year, the workshops placed only about 8 percent of 
their clients into competitive job's.; 

While important differences exist among workshops (including the 
degree to which they focus on transitional services), they have, as a 
group, been criticized for not doing more to help their workers enter the 
regular labor market; it is generally assumed by researchers and practi- 
tioners that many more retarded persons than how make that transition could 
do so if they were given appropriate training and support services. One 
reason frequently posed for the workshops' apparent reluctance to emphasize 
such services is their revenue needs, which usually depend on production 
contracts, which, in turf., depend on good workers* Another obstacle may be 
that funding from state rehabilitation agencies is not sufficiently 
flexible to cover intensive job placement efforts by the workshops 

(Whitehead, 1979: 19); 

} 

The state-federal Vocational Rehabilitation system is another source 
of employment assistance for retarded and other disabled persons. To be 
eligible for aid, applicants must have a physical or mental disability that 
constitutes a major handicap to employment, but not so severe that there is 
not a reasonable likelihood that the individual will f ind employment after 
receipt of the available services, primarily employment-related counseling. 
Other services include evaluation and assistance with training, job place- 
ment and transportation, often provided by other public and private 
agencies to which clients have been .referred. 

Public schools also provide employment training, particularly since 
the passage of the Education for All Handicapped Children Act of 1975. 
Vocational services are generally viewed as part of the educational process 



for older -students. While it is difficult to generalize about the nature 
and quality of these school services, they are usually offered in the form 
of work-study programs in which academic instruction is supplemented by 
work experience, either in the school or in community businesses and 
agencies. 

Numerous other local organizations that provide work experience and 
job placement services to economically disadvantaged persons include the 
mentally retarded and other handicapped persons among their clients. Many 
of these programs were, in the past, CETA-funded, and. some now' use re- 
sources from the Job Training Partnership Act (JTPA). 8 ^ However, these 
programs more traditionally served only a small proportion of the mentally 
retarded population. 

Placement and training programs focusing more specifically on mentally 
retarded persons have included several small 4 university-based initiatives 
for severely disabled persons; 9 Various efforts on the part of the Human 
Resources Development Institute of the AFL-CIO; a statewide supported work 
program for retarded persons in Massachusetts, coordinated by the Bay State 
Skills Corporation; and a number of small-scale transitional programs in 
other communities. One national effQrt to facilitate competitive employ- 
ment is an on-the-job training program, .administered by the Association for 
Retarded Citizens of the United States, which places retarded persons into 
public and private sector jobs and subsidizes , their wages fo.r an eight-week 
period. Resources for training retarded and other handicapped persons in 
competitive worksites are, in addition, available to local programs through 
the federally-funded Projects With Industry program, which is administered 
by the Rehabilitation Services Administration of the Department of Educa- 
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Programs serving retarded persons may also draw upon the Targeted Jobs 
Tax Credit (TJTC) program to encourage private sector employers to hire 
their clients. Through TJTC, employers can receive a tax credit pver a 
two-year period for part of the wages paid to eligible employees. Mentally 
retarded individuals receiving SSI or those referred from vocational rehabi- 
litation programs are eligible for TJTC. 

However, it should be noted that while varied . types of employment 
assistance are available for this population (as seen in the discussion 
above), many of the projects are small-scale in nature and limited in 
scope. Further, it is not clear how effective these programs have been in 
meeting the needs of this group, particularly because most studies of 
current and past efforts did not use matched comparison, or randomly 
assigned control groups in assessing program effects. As a result, there 
has been no reliable investigation of the costs and benefits of most 
programs operating in the fielcf. There is also little information on the 
operational experiences of such programs.^ 

In response to this knowledge and s.ervice gap, STETS was designed as a 
transitional employment program of relatively large scale (for this popula- 
tion), offering a program of intensive services targeted to individuals 
with a broad range of retardation levels. Its extensive research design 
makes STETS unique among other efforts aimed to assist and learn more about 
a population of mentally retarded young adults. 

III. The STETS Program Model 

The STETS demonstration grew out of the operating experience of the 
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National Supported Work Demonstration, an extensive five-year test of a 
transitional employment program that stressed work experience in a struc- 
tured, closely supervised work environment f or ' four groups of hard-to- 
employ individuals: long-term recipients of Aid to Families with Dependent 
Children (AFDC) , ex-offenders, ex-drug addicts and young school dropouts, 
inany with criminal records. 

MDRC held overall responsibility for managing this demonstration, 
which was rigorously evaluated. The analysis compared the experiences and 
behavior of an experimental group, which was offered Supported Work, to 
those of a randomly assigned control group, not offered the treatment. The 
study found that the. program was a cost-effective approach for two target 
groups, long-term recipients of welfare and ex-addicts. At 27 months after 
enrollment, welfare women who had taken part in the program were twice. as 
likely to have left the welfare rolls as women in a randomly selected 
control group. Also, earnings of participants had increased over those of 
control group members by approximately 50 percent. 11 

• Supported Work was also tried on a small scale for mentally retarded 
persons. Pilot projects were operated, by the Vera Institute of Justice in 
New York City and by Transitional Employment Enterprises in Boston, 
Massachusetts. While the mentally retarded group was not studied by MDRC, 
as part of the original Supported Work evaluation, performance measures and 
a separate, small impact study by the Vera Institute suggested the 
feasibility of this approach for the mentaLly retarded. As a result, it 
was decided in 1980 to test on a .larger .scale the effectiveness of a 
Supported Work variatioiu-for the mentally retarded. 

In designing the STETS model, MDRC staff members focused , on the basic 



.principles of the Supported Work program — gradually increased perform- 
ance expectations , close supervision and peer support. In considering the 
best way to assist mentally retarded persons through these techniques, MDRC 
staff were guided by the advice of a number of professionals in the field 
and by a committee composed of MDRC Board members and others with an 

interest and expertise in this area. Members of the advisory committee 

. . 12 

included representatives of the business, labor and academic communities. 
The main features of the STETS approach are summarized below, 
A, Target Population 

MDRC was interested in testing the STETS model for a broad group of 
the mentally retarded to examine program effectiveness for persons with 
different degrees of functional ability, A youthful population was chosen 
because past research had suggested that the employment difficulties of the 
retarded are most severe for teenagers and young adults (Coniey, 1973),. At 
that point in their lives, youths are moving from school to work, and dis- 
abled persons become ineligible for a number of support services they had 
previously received. Youths eligible for STETS were thus applicants 

between 18 and 24 years of age, with the primary disability of merual retar- 

. 13 
dation, generally in the moderate, mild or lower borderline ranges; 

Moreover, they were persons with very, limited, employment experience, 

(Eligibility criteria are discussed in more detail in Chapter II,) 

Additionally, the sites were discouraged from screening program applicants 

on the basis of judgments about the applicants' likelihood of success in 

the program. Few precedents for making, such judgments exist, and it was 

hoped that the STETS research would provide some lessons on precisely this 

issue. 
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B. Program Stages 

The program was structured into three sequential phases. 

Phase 1: Assessment and Work Readiness Training . Phase 1 of the pro- 
gram, * limited to 500 hours of paid employment, combined training and 
support services in a low-stress environment to begin developing in parti- 
cipants the basic work habits, skills and attitudes needed for their place- 
ment into more demanding work settings. This preliminary stage could take 
# place in both sheltered and. non-sheltered work, settings., but in all cases, 
Phase 1 participant wages were fully paid by the program. 

Phase 1 enabled program operators to assess the abilities and -inter- 
ests of retarded workers, a process considered essential for identifying 
needed support and training services. It also assisted staff in their 
efforts to place participants into appropriate jobs in subsequent phases of 
the program. Participants were to be engaged in at least 20 hours of 
weekly productive work, with additional time spent in other activities con- 
t^ibuting to the development of the behavior and knowledge required by 
-Phase Z positions. 

Phase 2: Transitional Jobs . At all sites. Phase 2 was essentially a 
period of "on-the-job 11 training in local firms and agencies. During this 
stage , participants were placed into non-sheltered positions where the 
stress and responsibility were to approach the level found in competitive 
jobs. Placements were emphasized that would lead to regular employment , 
and participants were to work. at least 30 hours per week. Wages were paid 
by either the program or employers, and in some cases, by a combination of 
the two. The STETS program supplied counseling and other support, services, 
and assisted the line supervisors at the host company in training and 
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monitoring. 

Because STETS was to provide a relatively quick transition to employ- 
ment, MDRC guidelines limited paid participation to 12 months during Phases 
1 and 2 ^combined. To allow for periods of inactivity caused by health, per- 
sonal or other pipoblfems, participation could span a 15-month calendar 
period, ~ * 

Phase y{ Post-Placement Support Services , The third and f ina 1 s tage 
was to begin after participants had secured regular employment. According 
to MDRC guidelines, Phase 3 started when each of the following conditions 
was met: ' * 

1) The employer was not receiving a financial subsidy from the 
program; ' 

2) The program had substantially reduced counseling and other 
services to both participant and employer; and 

3) The participant was considered by the staff and the employer 
-to be a regular member of the workforce, rather than a 

trainee. 

To ensure an orderly transition to work, the program provided up to 
six months of post-placement support services, tracked the progress of 
participants, and , if necessary, developed linkages with other local 
service agencies. 

IV. The STETS Sites 

As managing agent for the demonstration, MDRC selected local program 
operators to implement STETS keeping certain research considerations in 
mind* Diverse geographic locations, as well as varied organizational 
structures and 'methods of operation at the local level, were all needed to 
study program feasibility in different settings. The five organizations, 



local operating 



demons t ration's 
half of their 



chosen included a sheltered workshop in Cincinnati, |an ''affirmative 
industry" (which resembles a sheltered workshop) in St.* Paul, a State 
agency in Tucson, and two nonprofit training programs, one j in New York and 
the other in Los Angeles*. Each site vas to serve 40 to 55 ^ participants at 
any one time, a relatively high slot level for programs working with this 
population. 

Another important criterion was the capacity of each sponsoring agency 
to obtain local funding to support a large portion, of the 
costs. This factor was considered necessary as a measure of commitment to 
STETS, both by the local organization and the community, since local 
funding would eventually be needed to support STETS at the 
conclusion. Sites were, in fact, able to. generate over 
revenues from local sources. 

The five participating organizations are briefly described below 

A. Job Path (Hew York. Mew York) . 

STETS in Hew York was the only site sponsored by pn organization 
experienced in transitional employment programs for this population. Job 
Path is operated by the Vera Institute of Justice, a pxjivate nonprofit 
corporation established in 1961 to promote fairness and efficiency in the 
criminal justice system. Vera later expanded its activities to training 
and social services for a variety of disadvantaged groups. It pioneered in 
providing Supported Work for hard-to-employ populations, and in 1978 began 
Job Path. as a Supported Work program for the mentally retarded. 

While Job Path and STETS both use similar techniques, there are some 
distinctions. Job Path serves mentally retarded persons of all ages (the 
average participant is 28 years old), while STETS concentrated oh 18- to 
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24-year-olds, bringing an increased emphasis on a youthful population to 

n 

the organization. Job Path also differs from STETS in the length of time 
participants are allowed to spend in each program phase, 

B. STAR Center (Cincinnati. Ohio) , 

STAR Center (an acronym for Services, Training .and Rehabilitation) is 
the largest sheltered workshop in Cincinnati. It was established in 1972 
under grants from the Ohio Rehabilitation Services Commission and the 
Hamilton County Council for Retarded Citizens, Inc. In 1976, STAR Center 
became a division of the Workshops for Retarded Citizens, Inc. , which is 
also the parent organization for a job placement Service for the disabled, 
known as the Joy Center. 

STAR provides vocational evaluation, skills and job-readiness training 
and sheltered employment for disabled persons, with a focus on the mentally 
retarded and emotionally disturbed. In recent years, STAR has participated 
in a Projects With Industry program, making use of training positions in 
Cincinnati's largest hospital. Although STAR had previously attempted to 
prepare its clients for unsheltered positions, the emphasis placed on 
achieving this goal was increased considerably in the STEXS program. 

C. Minnesota Diversified Industries (St. Paul. Minnesota). 

Minnesota Diversified Industries (MDl) is a private, nonprofit cor- 
poration, describing itself as an "affirmative industry." It has provided 
employment and training to mentally retarded workers since 1964 in its St.. 
Paul plant, which has 130,000 square feet of manufacturing and warehousing 
space. When STETS began, MDI employed over 300 workers (most, but not all, 
of whom were disabled) in mainly light manufacturing, assembly, packaging 
and similar jobs under subcontract from several companies in the Twin 
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Cities area. 

While its facility resembles a sheltered workshop in some respects, 
MDI appears to emphasize more sophisticated production techniques than are 
typically found in workshops. Before its association with STETS, MDI had 
experimented with a few smaller-scale efforts to place retarded workers 
into unsubsidized jobs with local firms and agencies. 

D. Department of Economic Security^ Division of Developmental 
Disabilities (Tucson-. Arizona) . 

The only STETS project operated by a government agency was sponsored 
by the Arizona Department of Economic Security's Division of Developmental 
Disabilities (DDD) . This agency offers a variety of residential, voca- 
tional and support services to developmental ly disabled individuals of all 
ages. Many of its clients receive employment and training in a range of 
settings, including work activity centers, sheltered workshops and, increas- 
ingly* competitive employment. 

DDD does not itself offer training services, but typically coordinates 
services provided by other organizations and refers clients to them. 
Developmentally disabled individuals are assigned to a DDD "case manager/ 1 
who prepares an individual development plan, specifying the client's needs 
and goals, and strategies for achieving them. In STETS, J)DD would assume 
more direct responsibility for providing employment and training services 
to its clients. 

E. California Institute on Human Services and ADEPT (Los Angeles» 
California) 

The Los Angeles STETS program was run cooperatively by the California 
Institute on Human Services (CIHS) and ADEPT (Assisting the Disabled with 
Employment, Placement and Training). CIHS is an affiliate of Sonoma State 



University, which administers a variety of research and development 
project! throughout* California. In the past few yean, the organization 
has worked closely with state agencies, such as the California Department 
of Rehabilitation, on projects serving the mentally retarded and other 
disabled individuals. 

ADEPT is a nonprofit agency which was formed in 1974 by a group of 
individuals concerned about the underemployment of the disabled. It offers 
a variety of employment services to disabled workers, although most of its 

efforts to date have involved job placement %or physically handicapped 

// 

persons. CIHS was to exercise administrative, control over the Los Angeles 
STETS project, while ADEPT would provide 7 {he direct services to partici- 
pants. / 

The STETS program in Los Angeles operated in two separate ADEPT 
offices, the main one located in Panoroma City in the southern part of the 
San Fernando Valley. A satellite office was housed in an ADEPT branch, in 
downtown Los Angeles. 

Vv The Research Plan for STETS 

As a demonstration project, STETS incorporates a comprehensive 
research plan to evaluate the implementation experiences of the local 
sites, the impact of the program on enrollees, and the program's benefits 
and costs. The implementation study, the subject of this report, has been 

conducted by MDRC. The impact *nd benefit-cost analyses are being carried 

e 

but by Mathematica Policy Research (MPR) under MDRC'a supervision. 
A. The Impact Analysis 

The impact analysis represents an unusual opportunity to make a 



substantial contribution to the limited body of research on the effective- 
ness of employment programs for retarded persons. Unlike most previous 
studies, the analysis uses an experimental design: 496 STETS applicants 
were randomly assigned either to an experimental group, in which members 
were allowed to enroll in the program, or to a control gr' \p, where members 
could not enroll but could participate in other programs available to them 
in the community. 

Data for the impact study have been collected primarily through 
in-person interviews with members of both the experimental and control 
groups, and yith their parents, guardians or .other close acquaintances (who 
served as proxies when more complete or accurate information was needed). 
The interviews were administered at four points: at baseline (shortly 
after random assignment) and at six months (for two-thirds of the sample), 
1.5 months and 22 months after the baseline interview. 

The impact evaluation will focus on four broad sets of outcomes: labor 
market success, the use of alternative training and education services, the 
use of transfer programs and changes in life-styles. Labor market outcomes 
include such factors as employment, earnings, labor force participation, 
job search activity and employment stability. Alternative service out- 
comes include participation in various educational programs (e.g., regular 
high school, special education, work/study or vocational programs) , shelter- 
ed workshops and other employment programs. Transfer program outcomes 
include the receipt of SSI, AFDC and Medicaid benefits. Life-style out- 
comes cover three aspects of community and social adjustment: living 
arrangements, social skills (including the handling of money, budgeting 
skills and the ability to travel) and family formation. 

/ 
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B. Benefit-Cost Analysis 

The benefit-cost analysis will explore how the economic benefits -of 
STETS compare to the costs incurred in the program's operation. Building 
on the design of the Supported Work benefit-cost analysis of other dis- 
advantaged pjpulations, the research will derive a measure of the program's 
,f * let ..Present jyalue;" that is, it compares the present value of STETS bene- 
fits with program costs, and it does so from the perspectives of society in 
general, taxpayers and the participants themselves. 

One advantage of this study over previous analyses of employment pro- 
grams for the disabled is the use of a control group, which should provide 
more accurate daU on program effects. Moreover, because both samples are 
followed for a considerable period of time, program benefits can be assess- 
ed over a longer term than is usually available. 

C. Implementation Analysis 

The implementation analysis describes program operations at the local 
level and examines the factors influencing the sites' ability to recruit, 
train and place participants into competitive jobs. While MDRC's guidelines 
outlined the basic practices of the program, the sites were given wide dis- 
cretion in translating these guidelines into an operating program. The 
implementation analysis will assess the extent to which these guidelines 
were followed, and whether or not they were feasible, especially at the 
relatively high slot capacity at which each site was expected to operate. 
The implementation discussion will alsp help to interpret the findings of 
the impact and benef it~cost analyses. 

The implementation study seeks in addition to elucidate the operation- 
al advantages and disadvantages associated with alternative recruitment, 
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training and job development strategies, as well as the implications of 
various local conditions at the sites. This analysis .will be particularly 
important to any future efforts to* replicate STETS. 

A variety of data sources were used in this analysis. Over the course 
of the demonstration, qualitative data were collected by MDRC researchers 
In^LlL^yjLsiti. in which they interviewed STETS. staff and members of the 
organizations sponsoring the program, as well 'as a number of referral 
agents and employers. Ongoing program activities, such as worksite train- 
ing and group counseling sessions, were also observed. In addition, MDRC 
field representatives prepared progress reports after monthly site visits 
and reported on specific research issues. 

A major source o£ quantitative data in this report is the STETS 
Management Information System (MIS). Through a set of standardized forms 
completed at each site^ and regularly submitted to MDRC, demographic data 
were compiled on all enrollees (and control group members) at intake, the 
services received were recorded, and each person's progress through the 
program was tracked. Information was also gathered on the characteristics 
ot the worksites and the jobs into which participants were placed, both as 
trainees and regular workers. 

Additional quantitative data on employers were available from two 
special studies. In the job development contact study, program staff mem- 
bers kept a record, for a limited period, of their contacts with employers 
and the results of each contact. This study clarified the nature of the 
job development process and the level of effort required to place parti- 
cipants into jobs. The second study, a survey of Phase 2 employers, was an 
attempt to learn why employers had agreed to sponsor participants and how 
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they assessed both the STETS program and participants" performance. 

VI. The Context of the Demonstration 

Both the economic and funding contexts in which the program operated 
have implications for the lessons to be drawn from the STETS demonstration. 
Frori an economic perspective, STETS was implemented during a very severe 
national recession, which impeded job development efforts. A scarcity of 
jobs, in turn, adversely affected other parts of the program, as did* the 
persistent funding uncertainties, which disrupted planning, lowered staff 
morale and limited the life-span of the demonstration. Indeed, not long 
after the sites had begun to solve start-up problems, they were told to 
plan, for the program's phasedown. Consequently, there was relatively 
little time in which to refine training and job development techniques or 
to build a positive reputation with employers^ a prerequisite to successful 
job development. 

One implication of the shortened time-span should be considered in 
terms of the research results: the ( STETS evaluation may not reflect the 
ultimate potential of the program. It is possible that the implementation 
and impact studies may underestimate STETS accomplishments that might have 
emerged had the model been operated under more favorable conditions and 
allowed to mature. Yet, whatever the true potential, these findings and 
the forthcoming impact and benefit-cost analyses should provide useful 
information on this approach. 
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CHAPTER II 
RECRUITMENT. REFERRAL AND INTAKE 

* * "* » 

The first task facing STETS operators was to locate and enroll in 
sufficient numbers the types of individuals the .program was designed to 
serve. At each site, staff had to establish referral linkages with other 
agencies in contact wi>th members of the eligible population. 

This task was a challenge for two reasons. STETS was a new program, 
and staff at potential referral agencies had to be convinced that this 
program could help mentally retarded individuals. Moreover, the STETS 
research procedures were new requirements for both clients and referral 
agents and might possibly discourage referrals. 

This chapter (describes first the eligibility criteria and recruitment 
goals of the demonstration. It then reviews the, recruitment strategies and 
the referral and intake process, paying particular attention to the kinds 
of individuals enrolled in STETS. 

• ■* 
I- Eligibility Criteria and Recruitment Goals 

According to the demonstration's guidelines, individuals were con- 
sidered eligible for the program if they met the following criteria: 
• Age between 18 and 24. inclusive . As noted earlier, this 
age group was chosen in order to serve young adults who were 
* preparing, for or undergoing the transition from school to 
work or 'other activities. 
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• Mental retardation in the moderate to lover borderline 
range . F,or referral sources using IQ, scores, a range was 
selected* between 40 and 80; other verifiable determinants of 
mental retardation were also acceptable* 

• Unsubsidized full-time employment experience of less than 
six consecutive months in the two years preceding intake, 
and no employment of more than 10 hours a week in an unsubsi- 
dized job at the time of intake . This criterion was estab- 

, lished to limit enrollment to persons who would be likely to 
need the intensive employment services envisioned in the 
model. 

• Ho secondary disability tfait^in ^ttu^ refer- 
ral agency or .pitfgram operator, would make on-the-job train- 
ing for competitive employment impractical . While the 
demonstration was designed to test the program's effective- 
ness for a disabled population, it was recognized that, in- 
some cases, there would be^ secondary disabilities of such 
severity that individuals could not be expected to work 

# 

independently in a regular job. The sites were encouraged, 
however, to apply this standard only in exceptional cases* 
JLocal agencies in contact with the STETS target population were 
instructed to use these criteria in making referrals. STETS staff later 
verified eligibility in interviews with applicants, and at that point, 
randomly assigned the eligible applicants* 

New York, St. Paul and Tucson were to serve 50 participants at any one 
time, while Cincinnati and Los Angeles Were^to maintain slot levels of 40 
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and 55 clients, respectively. Site size was dictated in part by* the .sample 
needs of the impact analysis, but' planners also, wanted^to test the 
feasibility of the program v on a reasonably largef scale. Moreover, because 
half of the applicants were to be assigned to the control group, the sites 
-had to recruit twice as man/.individuals as the slot Levels specified, 
which increased the challenge d£ recruiting a sufficient number of 

applicants. — 

^ V 

Another^e^«txement related to the enrollees' level of retardation: 
JlD£C-<JCouraged sites to recruit half of their total- number from the 40 to 
60 IQ range in order to study the suitability of STETS for a divqrse popula- 
tion. This- criterion also guarded against a tendency^of referral sources 
to favor higher-functioning clients. 

II. Recruitment and Intake 

A. Recruitment Strategies 

Each site was given wide latitude in devising strategies to meet its 
recruitment goals. However, the similarity of approaches among sites is 
more striking than the differences. For example, all sites informed locaL 
agencies about STETS through mass mailings, but tended to emphasize 
personal presentations, usually given by the site directors. In these 
presentations, directors highlighted the positive reputations of their 
sponsoring agencies in working with handicapped persons. Larger sites also, 
used radio and television spots. Only two of the sites, Los rAngeles and 
New York, retained specialized recruitment staff to .assist in the in-person 
presentation of STETS. In Los Angeles, a "consultant was hired for three 
months when referrals were slow, and in New.,. York, a staff person who 
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handled public relations for the sponsoring agency assisted in the 
recruitment drive. * 
* " Screening Prior to Referral * 

The four eligibility criteria described earlier Were .the primary 
guidelines used by referral agencies. Interviews indicated that some 
referral, agencies did little or no further screening, but in other cases, 
more subjective criteria came into play. For example, certain agents 
referred only clients who had demonstrated a readiness and motivation to 
work thorough their past behavior and performance in school or training 
programs. Others gave priority to their higher-functioning clients, 
probably to see how well these persons fared before taking a chance on 
lowerrf unctioning individuals. Particularly in New York, agents were 
guided by the STETS staff in referring the kinds of individuals they 
believed the program would be most likely to help. However, while it seems 
clear that some screening took place, it is hot possible to know how 
widespread the practice was. 

It is also important to note that, in a minority of cases, eligible 
clients or their parents would not consent to a STETS referral when it was 
offered. While the reasons were not always clear, clients or parents were 
sometimes reluctant to give up places in other programs, such as sheltered 
workshops, when re-entry to those programs was not assured if they did not 
do well in STETS. Other clients or parents may have feared the loss of SSI 
or SSDI benefits, or have been concerned about re-establishing^ eligibility 
in the event of a job problem. It is not known from the data precisely how 
many clients or parents were affected by these or ojther factors, but 
referral agents and staff members interviewed indicated that few such cases 
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were identified by them. 

C . Screening After Referral - 

After referral, an applicant was interviewed by a STETS staff member 
whose primary responsibilities were to verify eligibility, explain the 
demonstration's experimental, design, and obtain the applicant's informed 
consent before random assignment. Typically, these intake interviews with 
applicants were scheduled with either a parent or the .referral counselor, 
or both, in attendance. This was considered important to avoid possible 
confusion over randoin assignment and to provide general support to the 
applicant. In some instances, however, applicants came alone. While this* 
was generally acceptable, referral sources were told that applicants who * 
might be unable to give informed consent should always be accompanied by* a 
person who could consent on thei : r behalf. " 

MDRC interviews with STETS staff and direct observations of the intake 
process indicate that most sites did little in the way of screening appli- 
cants beyond re-establishing the fact that they met the four basic eligi- 
bility criteria. File t searches and information from the site reports to 
MDRC show that the sites generally screened out only between 2 and 6 per- 
cent of their applicants for reasons other than inability to meet the objec- 
tive criteria. 

All sites mentioned physical disabilities (such as deafness), uncon- 
trolled seizures and mental illness as secondary disabilities which, in 
their judgment, might preclude competitive employment. Most sites, however, 
were willing to accept a fairly wide range of applicants. For example, Los 
Angeles relied almost completely on the judgment of referral sources, and 
made no further assessment. Tucson was relatively careful to discuss the 
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severity of secondary disabilities with referral counselors, but did not 
probe further. Cincinnati and St. Paul rejected very few applicants 
(although they did try to uncover serious illnesses not noted by the refer- 
ral sources) • 

New York was the exception; Job Path staff screened extensively on a 
number of criteria, turning down at least 20 percent of referred appli- 
cants. Program reports, observations and staff interviews suggest that 
most persons excluded from Job' Path for reasons other than inability to 
meet objective criteria had emotional problems, lacked motivation, or were 
inflexible in the kinds of training positions they were willing to accept. 
Job Path staff emphasized, however, that while they relied on their past 
experience with this group, it was quite often impossible to detect 
problems at intake. 

Random Assignment 

Random assignment and informed consent, the final stage of the intake 
process, were relatively new procedures for the sponsoring organizations. 
To ensure consistency, accuracy and sensitivity, all explanations of random 
assignment were based on material provided by MDRC, and a single staff 
person was given responsibility for the procedure. 

» 

After the program had been explained and eligibility determined, the 
intake worker discussed, in simple terms, the reason for random assignment 
and the meaning of being selected for the experimental or control group. 
The risks associated with holding a job, such as the potential reduction' or 
loss of benefits, were pointed out carefully. Applicants were also told 
that experimentals and controls would be interviewed, but that the 
interviews were confidential and could be refused , either in whole or in 



part, at any time. The intake person repeatedly asked if the applicant had 
questions , and explanations continued until the applicant and interviewer 
felt satisfied. At that point,, the intake worker telephoned the research 
firm to find out the applicant's assignment. 

Although it was sometimes difficult to gauge applicants' reaction to 
this process, MDRC interviews and observations suggest that most applicants 
assigned to control status were understandably disappointed, but were able 
to comprehend the neutral nature of the procedure; they did not appear to 
blame themselves. All control group members were sent back to their 
referral agencies, where alternative services could be sought. 

III. Number and Characteristics of STETS Enrollees 

Random assignment was phased in gradually, with the first site (New 
York) beginning in October 1981. The process was under way in all sites by 
early 1982 and was completed by December of that year. While the original 
research design had called for a sample of 500 each of experimental^ and 
controls, funding constraints caused a reduction in numbers so that, during 
the relatively short pericd of random assignment, 251 individuals were 
assigned to the program, and 245 to the control group. An additional 33 
persons, who had enrolled in the program before random assignment, will not 
be included in the impact sample, but are considered in this implementation 
analysis. The fact that 529 persons entered the program or the control 
group indicated that sites were able to recruit a substantial number of 
clients during a relatively, short period. 

Table 2.1 shows selected demographic characteristics of the enroLlees 
at the time of program entry. The population was a young one, averaging 20 
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DEMOGRAPHIC CHARACTERISTICS OF STETS ENROLLEES 
AT TIME OF ENROLLMENT, BY PROGRAM SITE 



Characteristic 


Los 
Angeles 


New 
York 


Cin- 
cinnati 


St. Paul 


Tucson 


All 
Sites 


Age (%) 












* t 

i 


18-21,. Years 


75 9 


fiO 5 


7 0 1 


& S O 

tO .2 


£. ft / 


67 .4 


22-24 Years 


24.1 


37.5 


27 .9 


34.8 


39.6 


32.6 


Average Age (in Years) 


20 


20 


20 


21 


20. 


20 


Retardation Range (%) * 














Moderate 


8.6 


4.7 


17V7- 


13.3 


20.8 ' 


12.8 


Mild 


48.3 


39.-1 


56.5 


55.6 


50.9 


49.6 


Borderline 


43.1 


56 .3 


25.8 


31.1 


• 28.3 


37.6 


Average Full-Scale IQ 














(Score) 


65 


68 


61 


63 


61 


64 


Secondary Handicaps (Z) a 














Any 


31.0 


41 .5 


37.1 


37.0 


60.4 


41.2 


Visual 


6.9 


7 .7 


4.8 


0.0 


11.3 


£.3" 


Seizure Disorder 


1 7 


u .Z 




ISO 


10 o 

1 J .2- - 


7 .0 


Cerebral Palsy 


A -9 


7.7 


0.0 


2.2 


3.8 


4.2 


Mobility Limitation 


3.4 


0.0 


3.2 


0:0 


11 .3 


3.5 


Emo tlo.nal 


0.0 


6.2 


6.5 


0.0 


9.4 


4.6 


Learning Disability 


8.6 


6.2 


1.6 


10.9 


7.5 


6.7 


Speech/ Language 














Disability 


ft fi 


n n 

\J .u 


O 7 

7- mi 


1 ft Q 


11 .3 


7 .7 


Other 6 


10.3 


20 '.0 


22.6 


13 .0 * 


13:2 


16.9 


Sex (%) 














Male 


60.3 


57 .8 


74.2 


56.5 


47.2 


59.7 


Female 


39.7 


42.2 


25.8 


43.5 


52:8 


40.3 


Ethnicity (%) 














White 


46.6 


31 .3 


45.2 


93 .5 


73.6 


55.5 


Black 


24.1 


50.0 


54.8 


6.5 


5.7 


30.4 


Hispanic 


22.4 


15.6 


0.0 


0.0 


20 .8 


12.0 


Other 


6.9 


3.1 


0.0 


0.0 


.0.0 


'2.1 


Current Living 














Arrangements (%) 














Parents/Relatives 


77.6 


87 .7 


88.7 


50.0 


67.9 


76.1 


Independent 


13.8 


3.1 


1.6 


15.2 


11.3 


8.5 - 


Group Home 


5.2 


4.6 


1.6 


28.3 


3.8 


7.7 


Other * 


3.4 


3.1 


8.1 


•6 .5 


16.9 


7,4 



(continued) 
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Los 


New 


Cin- 






All 


Characteristic 


Angeles 


York 


cinnati 


St. Paul 


Tucson 


Sites 


Benefits Received (Z) 














AFDC 




1 .0 


1 A 
1.0 




u .0 


1 O 
1 .0 


SSI 


24.1 


21.5 


27 .A 


13.0 


A1.5 


25.7 


SSDI 


6.9 


0.0 


3.2 


8.7 


11.3 


5.6 


General Asst. 


1.7 


3.1 


A.8 


13.0 


0.0 


, A. 2 


Food Stamps 


1.7 


0.0 . 


A.8 


A. 3 


13 .2 


A. 6 


Medicaid 


15.5 


27 .7 


A.8 


A7 .8 


3.8 


19.0 


Schooling (Ever 














At ten&od Since Age 1 4) 














<*)* 














Mainstream (includes 














Vocational) 


77.6 


' 55. A 


. 79.0 


69.6 


5A.7 


67.3 


Special (Only 














Handicapped) 


20.7 


A6.2 


35.5 


A1.3 


A1.5 


37.0 


Curriculum (Ever 












e 


Followed Since Age 14) 














/ m,\ a 

(%>- 














Regular Curriculum 


15.5 


9.2 


11.3 


10.9 


5.7 


10.6 


Special Curriculum 


79.3 


81 .5 


100.0, 


82.6 


86 .3 


86 .3 


Employment & Training 














Experience (%)* 














Mainstream Training 


13 .8 


L.5 


6.5 


21.7 


3.8 


8.8 


Special Training 


12.1 


33.8 


A.8 


65.2 


26 .A 


26 .8 


Urtsubsidized Employ- 














mpnt — Fill 1 Timo 


1 9 1 


10 1 


1 & 1 
10 .1 


1 / *H 


11 .3 


13../ 


Unsubsidized Employ- 














ment - ^Part-Time 


■ 25.9 


6 .2 


24.2 


41 3 


Q A 




Subsidized Employment 


AA.8 


43.1 


29.0 


17 .A 


11 .3 


30.3 


Sheltered Wor)c* 


6.9 


16,9 


3.2 


19.6 


26 .A 


1A.1 


Volunteer Work 


15.5 


10.8 


3.2 


2.2 


9. A 


8.5 


Number of Enrol lees 


58 


65 


62 


A6 


53 


28A 



SOURCE: MDRC tabulations of Application/Enrollment Forms in the STETS 
Information System. 

NOTES: Percentage distributions may not add exactly to 100.0 because 
of rounding. 

a 

More than one category may apply to an enrol lee. 

b This category includes hearing and general health impairment. 

c 

This reflects benefits received by the enrollee and does not 
necessarily include benefits received by someone else in the respondent's 
household. 



-29- 



52 



years of age (meaning that this group was subject to the employment 
difficulties found among many young people). Almost 60 percent of 
enrollees were male, a proportion similar to that in the general population 
of persons labeled retarded (Ingalls, 1978). About 56 percent of enrollees 
were white, 30 percent black and 12 percent Hispanic. With the exception 
of Cincinnati, and to a lesser degree New York, the ethnic distribution 
within local programs reflected the ethnic makeup of each site's community 
(U.S. Department of Commerce, 1980). 

Overall, 13 percent of the enrollees were classified as moderately 
retarded, about 50 percent fell into the mild range, and 38 percent were 
borderline. The program thus fell somewhat short of its goal in recruiting 
a substantial proportion of moderately retarded individuals. In retro- 
spect, this is not surprising; only 6 percent of all mentally, retarded 
persons fall into this category (Tarjan et al., 1973). Concern was also 
expressed by referral agencies and parents about this group's prospects for 
competitive employment. As a new program, STETS had no proven track record 
with which to respond. 

Tucson and Cincinnati were most successful in recruiting moderately 
retarded participants, probably because of the close associations of these 
sites with other agencies serving lower-functioning individuals. In 
Tucson, the STETS parent organization, the State Division of Developmental 
Disabilities, referred almost half of the program's enrollees in that 
range. In Cincinnati, a special county-based system serving individuals 
with IQs of 55 and below referred youths to STETS through the vocational 
rehabilitation system. 

Secondary handicaps were identifiable at intake for 41 percent of 
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enrollees. Since it has been estimated that a • many aa 50 percent of all 
retarded children with iQa below 70 have secondary handicaps (Conley, 
1973), this percentage is reasonably representative of the eligible popula- 
tion. However, since non-physical secondary handicaps were not always 
noted by either the referral source or the intake interviewer, the 41 
percent ^figure may understate the actual prevalence of disabilities among 
enrollees. 

The education and employment backgrounds of enrollees suggest other 
obstacles to future labor market success. From the age of 14, approxi- 
mately one-third had never attended a regular school (including vocational 
or trade schools), and 89 percent had never participated in a mainstream, 
as opposed to a special education, curriculum. Similarly, while 27 percent 
had had some vocational training (outside of school) in a special training 
program 'for disabled persons, a very small group, 9 percent, had partici- 
pated in a mainstream vocational training program. Moreover, only 14 per- 
cent had ever worked in a full-time unsubsidized job while only one-fifth 
had been employed in a part-time unsubsidized position. As would be expect- 
ed with a young, handicapped population with little previous employment, 
over three-quarters of enrollees were living with parents or relatives. 

Although STETS imposed no income-related criteria, a number of enroll- 
ees did receive benefits, or their families received benefits, from needs- 
based public assistance programs. As Table 2,1 indicates, 19 percent of 
the enrollees were receiving Medicaid* payments , and some 6 and 26 percent 
were receiving Social Security Disability Insurance (S',DI) and Supplemental 
Security Income (SSI), respectively, at intake. The Tucson site had the 
highest proportion of dependent enrollees; 42 percent received SSI bene- 
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fits. (At that s*ie, recruitment efforts focused on a pool of individuals 
among whom SSI recipients were heavily represented.)* 

Since SSI eligibility could be lost,, after a nine-month "trial work 
period," recipients who joined STETS risked the loss of rather substantial 
benefits, particularly since SSI receipt also qualified an individual for 
Medicaid and, in some cases, Food Stamps. The fact that more than a 
quarter of the enrollees were receiving SSI suggests that, despite the 
risk, recipients were not necessarily adverse to employment opportunities. 
The question of whether these recipients can successfully become self- 
supporting through an intervention like STETS is one of considerable inter- 
est to society, and will be addressed in the. impact study. Unfortunately, 
there are no data on how willing SSI recipients were to enroll in the STETS 
program initially. 

IV. Referral Sources 

Table 2.2 shows that state vocational rehabilitation agencies and 
public schools were the principal referral sources for STETS, followed by 
state/county agencies for the mentally retarded and sheltered workshops. 
STETS sponsoring organizations already had referral linkages with voca- 
tional rehabilitation agencies, but their linkages with schools were much 
weaker. The young age criterion for STETS made it necessary for program 
operators to build this relationship quickly. 

Schools generally were enthusiastic about the program,, but there was 
some uncertainty initially about whether they would refer youths under 22 
yeafc of age. Under the Education for AH Handicapped Children Act, 
schools are mandated to serve interested handicapped youths until they 



TABLE 2.2 

PERCENTAGE DISTRIBUTION OF STETS ENROLLEES, 
BY TYPE OF REFERRAL AGENCY AND PROGRAM SITE 



Type of Referral Agency 


Los 
Angeles 


New 
York 


Cin- 
cinna ti 


St. Paul 


Tucson 


All 
Sites 


State Vocational 
Rehabilitation Agency 


34.5 


16.9 


46.8 


80 .4 


7.5 


35.6 . 


Public School System 


17.2 


35.4 


51.6 


0.0 


34.0 


•29.2 


Sheltered Workshops 


10. 3 


26.2 


0.0 


0.0 


20.8 


12.0 


State/County Agency for 
Developmental l.y Disabled 


20.7 


1.5 


1,6 


8.7 v 


37.7 


13.4 


Other 8 


•17 .2 


20 .0 


0.0 


10. .9 


0.0 


9.9 • 


Total 


100.0 


100.0 


100.0 


100.0 " 


100.0 


100 .0 


Number of Enrollees 


58 


65 


62 


46 


53 


284 



SOURCE: MDRC tabulations of Application/Enrollment Forms in the STETS 
Information System. 

NOTES: Percentage distributions may not add exactly to 100.0 because 
of rounding. 

a „ 
Includes CETA and various private non-profit organizations 
serving the handicapped. 
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reach that age. Only in St. Paul, however, did .youths more typically enter 
the demonstration after having finished school. In tact, in that site, an 
unusual recruitment pattern was observed. Although it appears in Table 2.2 
that no school referrals were made, vocational rehabilitation counselors 
outstationed in the schools referred a number of students to STEXS. 

While sheltered workshops were, like the rehabilitation agencies, a 
traditional referral source for the sites, the proportion of referrals from 
these agencies was not especially high. One possible explanation is that 
they needed good workers to maintain their own productivity levels, but the 
more critical factor was probably age. It was x ^ported that most workshpp 
employees were too^old for STETS. 

State or county mental retardation/developmental disability (MR/DD) 
agencies were particularly important referral screes in Tucson and Los 
Angeles, where 3.8 and 21 percent of enro-Llees, respectively, came from such 
agencies. These sources were also important in Cincinnati (although Table 
2.2 does'* not reflect this because referrals from those sources^ usually were 
funneled through the vocational ' rehabilitation system). As mentioned 
earlier, Cincinnati made a special effort to involve a countyr-based MR/DD 
system that provided school, residential and work activity services to 
individuals with IQs below 55. Although a linkage had long existed between 
this system and the STAR Center (an administrator sat on the STAR Center 
board of directors), referrals were few because of STAR's relatively high 
productivity standards. The STETS outreach to this and other MR/DD 
agencies represented a major change in referral relationships for the STAR 
Center. 
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V, Meeting Enrollment Goals 

While the sites overall were able to enroll a sizable, diverse and 
disadvantaged group, recruitment was not without problems,, and the sites 
did not reach their contracted slot levels. At four of the five sites, the 
average enrollment ranged from 35 to 43 individuals during the most stable 
period of operations, St, Paul, averaged only 30 enrollees during this 
study period. Two main problems were funding constraints and the difficul- 
ties of job development. In several of the sites, the pace of recruitment 
efforts slackened during the spring of 1982, when the long-term funding out- 
look, was unclear and job development was proceeding slowly. In some cases, 
random assignment and the paperwork requirements of the demonstration 
discouraged referral agents,, particularly at the beginning of the demon- 
stration. In all instances, it was difficult to restart recruitment 
efforts quickly when more funding became available. 

Another problem, particularly in New York, Cincinnati and St, Paul, 
was caused by the funding and staff reductions facing vocational rehabili- 
tation agencies during the demonstration period. These limited the ability 
of referral agencies to process and refer appropriate .clients to STETS • 
St. Paul had additional problems, including poor outreach -„ to facilities 
serving the lower-functioning, the apparent reluctance of neighboring 
counties to refer to programs outside their areas, and, as previously 
mentioned, the referral policy of the schools. The most important factor, 
however, was that site's slow pace of job development. In the words of the 
St. Paul project director, "If you can get the jobs, you can get the 
people. 11 _ 

Despite these difficulties, STETS — and the idea of competitive 



employment — held considerable appeal for referral agents, as well as for 
retarded persons and their parents. At the very least/ this is reflected 
~jj|_thejEact that well over 500 applications were made to the program during 
its short period of recruitment. 
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CHAPTER III 
TRAINING AND WORK EXPERIENCE 

For each phase of STETS, MDRC issued a broad set of guidelines for the 
sites to follow in operating the program. However, within those guidelines 
the sites were given much discretion to determine the specific content of 
training procedures and work experience. To understand STETS, it is thus 
necessary to understand how these activities were actually implemented, and 
some of the advantages and disadvantages of the alternative approaches 
which were tried. 

This chapter opens with an examination of the nature of the challenge 
-facing program operators as they began their training efforts. It speci- 
fies the typical difficulties that participants experienced as they per7 
formed* their jobs difficulties that the operators viewed as obstacles to 
permanent competitive employment. (For a further discussion of partici-r 
pants' work performance, see Chapter V.) The chapter then covers the three 
phases of STETS in some detail, presenting operational techniques and 
lessons which may be helpful in future replications of the. program. 

I. Deficiencies in Participants* Work Habits and Skills 

Given the nature of the target population, it is not surprising that 
most STETS participants experienced some difficulties at work. These in- 
eluded deficiencies in work habits (the generic behavior necessary for keep- 
ing almost any' type of job) r as well as in work skills (skills associated 

with a particular job). In each of these two, areas, staff interviews 

« 

suggest that the most common types of problems fell into several major cate- 
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gories, which are discussed in the following pages. It is impossible to 
specify from the data how many participants had these problems. It is 
clear., however, that most participants had one or some of; these difficul- 
ties, but not all. At the same time, many participants were able to over- 
come their limitations and eventually enter unsubsidized jobs. 
A. Work Habits 

1. Attendance and Punctuality . Irregular daily attendance (in- 
cluding lateness to work and in returning from breaks) was perhaps the 
most prevalent problem, particularly during Phase I. This could be caused 
by a number of factors: a low commitment to work, a tendency to oversleep, 
misjudging travel time, or difficulties in using public transportation. 
Sometimes participants just did not understand the appropriate reasons for 
missing work. As one counselor said of a participant: 

When he first started the program, he would keep taking days 
off whenever he wasn't feeling well. I tried to help him 
understand that he still has to go to work, even if he's not 
feeling great. 

Counselors also had ta encourage many participants to remember to call 
th^ir supervisors if they were going to be late or- absent. 

2. Interaction Skills . Some participants had problems getting 
along .with their supervisors and co-workers, and a difficulty common to 
many of them was an unwillingness to accept criticism. At one site, a 
supervisor complained that a participant threatened to quit whenever he was 
told that his performance was not up to standard. Some participants argued 
with supervisors and co-workers, or were disruptive in other ways. 

3. Concentration and Endurance . A number of participants had 
short attention spans and were easily distracted* problems which affected 
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both the quantity and quality of their production. Related difficulties 
were a lack of stamina or tolerance for a full day's work or accepting the 
importance of a consistent working routine. 

*• Completing. Initiating and C hanging Tasks , Participants were 
often not thorough in completing tasks or had difficulty switching tasks. 
Others would simply stop working when they had finished their assignments, 
rather than telling their supervisors they were available for other tasks, 
or finding them on their own. A Phase 2 employer offered these comments 
about one participant's chances for becoming competitively employed: 

He would have to become more enthusiastic about finding out 
what needs to be done. In hiring, I look for the person who; 
. is finding things that nead to be done, so they are creat- 
ing their own work and understanding their responsibility. 
I' think he could grow into that ~ instead of asking what 
needs to be cleaned, he'd go out and see what needs to be 
cleaned, and clean it. 

5. Grooming and Hygiene. Not all participants knew how to dress 
for work or understood the importance of keeping themselves clean, combing 
their hair ,attd maintaining proper hygiene. 

Overall, STETS staff members postulated that many of these poor work 
habits were more manifestations of general immaturity than of retardation, 
and some pointed out that the behavior problems were most pronounced among 
younger participants. In addition to immaturity (or perhaps related to. 
it), staff members Identified two other important sources of "poor work 
habits: lack of motivation and an insufficiently supportive home environ- 
ment. In several sites, particularly New York and Cincinnati, staff mem- 
bers found that poor work habits were common to participants whom they con- 
sidered both higher-functioning and "streets-wise. " Some may have been mis- 
labeled as retarded by school systems, and they understandably resented 
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being viewed as such. 

B. Inadequate Work Skills 

Staff members believed that the poor work habits of their clients 
usually presented greater obstacles to competitive employment than their 
limited work skills. Still, participants often did not possess certain 
basic competencies that are 'Critical for most entry-level jobs. 

Perhaps the most prevalent of these deficiencies was a lack of speed. 

Staff members reported that some participants simply did not work fast 

enough for many competitive jobs. One Phase 2 employer described in this 

way the activities of a participant who eventually had to be terminated 

from his position .at. a bakery: 

His job is primarily cleaning pots and pans, cleaning the floors, 
the machinery, and equipment. Actually, he should be able even- 
tually to help us with things like icing the cookies — but 
frankly, he's so slow he spends most of his time on pots and 
pans. He spends eight hours on a job that should take three 
< hours. For me, it's, not economical. 

Other participants had poor reading, writing and math skills; diffi- 
culty understanding or remembering how to do some of the assigned tasks; 
fprgetting to do all of the tasks; and forgetting to vary their work accord- 
ing to a set schedule.* For example, as one counselor npted, some partici- 
pants in janitorial positions had trouble remembering "that you clean some- 
thing on Wednesdays and Fridays only, and that you clean something else on 
other days. 11 

AH these deficiencies were the focus of staff attention during Phase 

2 

1. and often later in the training process. Strategies for addressing 
those problems are described next. 



II. Phase 1 

Fundamental to the STETS model was the concept of "graduated stress," 
where 'the job tasks and responsibilities imposed on trainees are at first 
kept low and then raised gradually to a- competitive level. This process 
began in Phase 1 when individuals were assigned to short-term, minimum- 
stress jobs, intended to introduce them to the norms of the working world 
and to help them develop the habits and attitudes required for successful 
employment. Also, during this period, staff members conducted an initial 
assessment of the kinds of training and support services each participant 
needed. In order to ensure that clients were moved expeditiously into more 
demanding Phase 2 training positions, Phase 1 employment was limited to ,500 
hours. However, participants had to work at least 20 hours per week during 
this time. 

Phased made use of two basic approaches, primarily because training 
was offered in two different types of. settings. The first approach (used 
in Cincinnati, St. Paul and Tucson), allowed for assessment and training of 
participants in a sheltered setting. The second (used in New York and Los 
Angeles) provided these services to participants who had been \placed "in 
real work settings, usually in public agencies or nonprofit organizations.. 
^ In the latter two sites, full subsidies were available for Phase 1 work, 
S$ primarily through CETA. Additionally, subsidies in Los. Angeles were avail- 
able through a state youth program. As the following discussion suggests, 
both the shelterecT and non-sheltered approaches had their advantages and 
drawbacks. 

A. Phase 1 in Non-Sheltered Worksites 

1. New York . While both the New York and Los Angeles sites used 
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the non-sheltered model, assessment and training were much more structured 



that interested them, and some of their past experiences. Subsequently, 
several staff members — the job developers, program director and counse- 
lors 3 — decided jointly on suitable worksites. To the extent possible, 
staff tried to arrange jobs according to participants' interests, but the 
range of available Phase 1 jobs was confined mostly to messenger, food- 
service, janitorial and light clerical positions. Table 3.1 presents the 
distribution of these Phase 1 jobs. 

Within these constraints, other factors in matching participants to 
jobs included: their willingness to work in certain positions,, their 
ability to do the work (for example, a person who could not read well was 
not assigned to a clerical position); the suitability of the supervisor and 
the setting for a given participant's personality (for instance, more 
supportive supervisors were matched to participants who required greater 
assistance); and the proximity of the worksite to the participant's home. 

Ojice a participant was placed, the assigned counselor visted the work- 
site usually twice a week, depending on the participant's needs, I he visit 
tculd be 10 minutes or as long as several hours, with the average visit 
lasting from 20 to 30 minutes. Training and monitoring were the respon- 
sibilities of the employers' regular worksite supervisors, many of whom had 
previously worked with mentally retarded clients of Job Path, the New York 
site's sponsoring agency. The counselors' work centered more on ensuring 
that the work experience met the enrollees' needs and that the skills devel- 
oped could be transferred to the private sector. As one counselor 



activities .in New York. There, assessment began at the first intake meet- 
ing, during which counselors observed applicants and asked them about work 
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TABLE 3-/1 

PERCENTAGE DISTRIBUTION OF TYPE OF fflRST JOB 
FOR PARTICIPANTS EVER ASSIGNED TO A 
JOB, BY PHASE AND PROGRAM SITE 



— J J — 

Phase and T lit 
Occupational Category 


Los 
Angeles 


New 
York 


Cin- 
cinnati 


St. Paul 


Tucson 


All 
Sites 


Phase 1 














Clerical md Sales* 


23.4 


58.5 


- 0.0 


11.9 


2.1 


21 .0 


Service 


29,8 


41.5 


0.0 


2.4 


4.2 


16.8 


Agriculture, Fish, 
Forest 


8.5 


0.0 


0 .0 


0 .0 


1 l 


1 Q 


Processing 


0.0 


0.0 


0.0 


0 .0 


4.2 


0 .8 


Machine Trades 


2.1 


0.0 


0 .0" 


76.2 


0 .0 


12 6 


Benchwork 


14.9 


0.0 


6.7 


0 .0 






Structural Work 


14.9 


0 0 




O It 


H .2 


3 .8 


* c 
Miscellaneous 


6.4 


0.0 


93 .3 


7 .1 


27 1 


o »o 


Total 


100.0 


100 .0 


100.0 • 


100 .0 


100.0 


inn n 


Number of Participants 


47 


65 


60 


42 


48 


262 d 


Phase 2 














Clerical and Sales 8 


20.6 


51 .0 


5.3 


0.0 


0.0 


19.9 


Service 


26.5 


47 .1 


65.8 


78.9 


61*8 


53 .4 


Agriculture, Fish, 
Forest 


11.8 


0.0 


0 .0 


0 ,0 


0 .0 




Processing 


2.9 


0.0 


5.3 


0 .0 


2 .9 


2 .3 


Machine Trades 


2.9 


0.0 


0.0 


5.3 


8.8 


2 .8 


Benchwork 


23.5 


0.0 


7.9 


15.8 


14.7 


10.8 


Structural Work 


2.9 


2.0 


0.0 


0.0 


0.0 


1.1 


Miscellaneous 0 


8.8 


0.0 


15.8 


0.0 


11.8 


7.4 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


Number of Participants 


34 


51 


38 


19 


34 


176 e 



(continued) 
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TABLE 3.1 (continued) 



SOURCE: MDRC tabulations 0 f Monthly Status Change Forms in th* STETS 
Information System, 



NOTES: Occupational categories were derived ftfom groupings of 
similar jobs, asjdefined by 1-digit codes from the Dictionary of 
Occupational Titles . 4th Edition, published by the IKS, Department of 
Labor, Employment "and Training Administration, in 1977. 

j 

Percentage distributions may not add .exactly to 100.0 
because of rounding. 

j. 

This category includes, but is not limited to, filing clerk, 
dat% clerk, mailroom clerk, and messenger. 



worker, kitchen 
c 



'This category includes, but is not limited to, fast food 
worker, housekeeper, porter, and janitor. 

This category primarily includes packaging jobs. 



Excludes one participant in Los Angeles and two in Tucson 
for whom data aije missing. 



e * 

Excludes three participants in Los Angeles, one in New York, 
two in Cincinnati, three in St. Paul and two in Tucson for whom data are 
missing. 



explained: 



You can't just leave it to the supervisor. You have to make 
sure what the trainees are learning. We try to find out 
what the private sector employers will expect, and make sure 
that the public sector supervisor is providing the appropri- 
ate training. 

Counselors also tried to stay informed about whether supervisors were 
holding participants to realistic performance standards, which could as 
easily be too low as too high: 

Sometimes they may not have high enough standards. We have 
to make sure the trainees are learning what they are 
supposed to be learning. Sometimes supervisors feel over- 
protective of trainees and don't want to tell the counselors 
about the trainees' problems. We have to urge the super- 
visors to share the problems the person is having so we can 
help them work on those problems. 

Added another counselor: 

At some sites the supervisors feel sorry for trainees and 
"baby" them. They say, "We don't want to push them too hard 
— we'll never have the problems they have. 11 But this 
doesn't help the trainee. 

Counselors also helped out with the actual task instruction. Often, 
for example, when a participant began a new job, the counselor also per- 
formed the tasks in order to understand job expectations and to communicate 
these clearly to the participant. -One counselor described this approach: 

I watch them work. ' Like Bill^ — I watched him wash 
tables. I saw he wasn't using much elbow grease. So I took 
the sponge and showed him how to dp it. With Edward, I help- 
ed to show him how to file faster. I may go with the messen- 
gers on their runic to show them how they could do their runs 
more efficiently. I work alongside them because we can't 
just rely on counseling sessions to find out what work 
problems they have. They may not say anything about them. 
Even in follow-up, I'll work alongside the person, but it 
happens more during Phase 1. 

Another counselor described how he assisted two participants who were 
mopping a hospital floor: 
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They were told to do one side first and then the next. They 
didn't understand this and did the whole floor. So 1 took 
the mop and showed them how to do it and where to put the 
sign (indicating that the floor was wet). 

Intervention was sometimes even more impprtant when the problem was 

not skills, but work habits: 

If someone's hygiene and grooming are bad, the supervisor 
will tell me,, and I'll have to deal with that. If there's 
something he feels funny about dealing with, that's where I 
pick up the slack — things that they may not usually have 
to deal with for other workers, 
< 

The counselors tried to talk to the supervisors whenever they visited 
the worksites — even if only briefly. They also endeavored to meet with 
them biweekly to go into more depth about participants' activities and 
performance. 

In addition to introducing the work world, Phase 1 was a time for 
early and careful assessment of work problems and training needs. 
Assessment was an ongoing process in Mew York, based on worksite 
observations or feedback from supervisors. Approximately six weeks after a 
youth had started Phase 1, counselors summarized the performance assessment 
in a "Habilitation Plan," which outlined the problems the participant 
needed to address and defined how the program would attempt to help. The 
plan was reviewed and, if necessary, revised every 90 days. 

Formal testing, which was administered in the Cincinnati and St. Paul 

sites, wac not part of New York's assessment process. As the project 

director explained: 

Assessment is done through actually seeing the person in a 
real work situation, not a .simulated situation. Work 
samples may tell about dexterity, but not other things. A 
key advantage of our approach is that you can see the kind 
of ambiance a person works well in, what kind of environment 
a person would be suitable for, and what the person's skills 



are. This approach, compared to the packaged testing 
schemes, brings the assessor closer to the person being 
assessed. .You get richer and more detailed information on 
people. 

2. Los Angeles . For participants in the Panorama City office of 
the Los Angeles STETS program, most Phase 1 placements were at the Veterans 
Administration Hospital in a neighboring city, which for several years had 
provided placements for clients of ADEPT, the STETS sponsoring agency; 
Positions available at the hospital included custodial, food service, 
internal messenger and mailroom jobs. The downtown STETS office obtain- 
ed similar positions at other hospitals. 

Other Phase 1 worksites included the California Conservation Corps, a 
for-profit candy factory and two sheltered workshops. Not many partici- 
pants were initially placed in the workshops, however, because workshop 
staff objected to paying STETS workers the minimum wage at a time when 
other workers received piece rates. However, the ST,NTS staff in Los 
Angeles became more favorably disposed toward the workshops by the end of 
the demonstration. One staff member described the change: 

Initially, our thinking was that the whole idea of STETS was 
to get people out of workshops. But after seeing our 
participants, we came to think that for some participants a 
workshop would be good for them at the beginning. Some 
needed a little more time to understand what work was all 
about. The idea of being productive. Meeting quotas. They 
get closer' monitoring at the workshops. 

As in New York, the Los Angeles program depended on the worksite super- 
visors to conduct most skills training. In contrast to New York, counse- 
lors primarily assessed performance through telephone contact with particir 
pants' supervisors rather than through direct observations of participants 
at their worksites. Worksite visits were limited usually to the one month- 
ly - 



ly visit required by CETA. Contact with participants was otherwise kept up 

by group counseling sessions at the program site and by telephone.* 

Also in contrast to New York, the program used outside consultants to 

teach some participants how to perform jobs. From the perspective of the 

Los Angeles director of client services, it was sensible to subcontract job 

tutoring when it was needed, rather than to depend on the counselors: 

None of us really has the knowledge to do what a real job 
tutor does in terms of breaking down tasks and teaching them 
to the participant. 

In the downtown office, one counselor noted: 

A job tutor on our staff was very useful for helping with 
the buses, and going around and constantly checking with the 
participants while they were working. I tried to do that, 
but I was so busy trying to develop jobs that I couldn't go 
back to the participants as much as I felt was needed. I 
think it is necessary to keep going back. 

The training process at the Los Angeles program was unfortunately 
impeded by the site's administrative arrangement. As noted earlier, ADEPT 
is primarily a placement agency for disabled persons (most of whom are 
physically disabled), and the organization had previously offered little in 
the way of worksite support services to its clients. According to the 
original plans for the STETS program, STETS staff were to provide th£ 
support services to STETS participants, with ADEPT counselors providing the 
placements. It proved difficult, however, for the two staffs to coordi- 
nate, and the arrangement was changed so that ADEPT staff handled both 
placement and support for one set of participants, while STETS staff per- 
formed the same functions for a second group. 

Throughout much of tfie demonstration, the level of expected ADEPT 
support did not materialize. Staff of this organization were particularly 
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slow to provide on-rthe-job training, the responsibility with which they 
ware least familiar. In the end, STETS participants were treated no differ- 
ently from ADEPT'S regular clients, and the Los Angeles site functioned 
more as a job placement agency than as the kind of Supported Work program 
envisioned in the original program design. 
B. Phase 1 in Sheltered Workshops , 

In Cincinnati and St. Paul, Phase 1 activities took place primarily in 
production facilities operated directly by the sponsoring organizations. 
The Cincinnati sheltered workshop had approximately 150 employees perform- 
ing assembly and packaging jobs, many of which required the use of 
machinery. Most of the tasks involved packaging games, toys and greeting 
cards in cellophane wrapping; and counting, bagging and sealing nuts and 
bolts, candies and other items. The organization had also acquired a 
plastics warehouse, where participants were called upon to sort plastics 
for recycling and to feed them into machines. The program, in addition, 
operated a janitorial service, in which a small crew, under the direction 
of a staff member, performed cleaning services for local businesses. 

While in many ways similar to a typical sheltered workshop, the St. 
Paul production facility differed in its composition of workers and types 
of jobs. Approximately 35 percent o£ .the facility's employees were 
non-handicapped persons who served as role models for the handicapped 
workers. Also, their higher productivity and skills allowed the 
rganization to bid on more sophisticated and varied production contracts. 
During the demonstration period, some of the jobs in this large facility 
included collating, inspecting and packaging postage stamps ; assembling 
small products, such as jacks for telephones and stereo equipment; 
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reconstructing wooden pallets; and disassembling old telephones for 
recycling. Many tasks involved the use of fairly complicated machinery. 

In Tucson, the third site using a sheltered model, an independent 
agency, The Handlers, Inc., provided Phase ,1 services during most of the 
demonstration. 3 Some participants were assigned to benchwork sorting, 
assembly and disassembly tasks in a sheltered workshop setting, where all 
of the tasks were relatively simple and required little use of machinery. 
Others were placed at a Handlers' woodworking facility, where they built 
crates and pai'lets, using power saws and other such machinery. 

In all three sites, training took place on the job, and except in 
Tucson, STETS counselors were in charge of it". As in New York, the 
counselors demonstrated specific tasks, observed workers' performance and 
offered the necessary criticism. 6 Much of the instruction entailed break- 
ing down tasks into simple components and demonstrating each step separ- 
ately, a process recommended by a number of experts in the field of mental 
retardation. 7 Praising participants for correct performance was an 
important part of this process, and it always took place within hearing of 
other workers. This no.t only encouraged the particular participant, but 
also highlighted for others the value of effective and efficient 
production. 

Typically, training in Cincinnati and St. Paul was most intensive 
during the first few days. At this point, it was not unusual for trainers 
to instruct and observe participants closely for several hours a day. This 
type of attention could usually be reduced within a few days unless a 
participant was shifted to a new job in the workshop (a frequent practice 
used to broaden experience and to meet changing production needs); when 
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such a shift occurred, the level of instruction intensified again, 

Cincinnati's training process^was particularly structured. There, 
trainers drew graphs wh?ch charted participants' productivity, attendance 
and punctuality, and shared them with participants. "Performance contracts" 
were developed* for problem workers to specify levels of improvement needed 
within set periods of time. Similar techniques were used in Tucson by the 
counselors employed by The Handlers, who conducted the training, STETS 
counselors visited participants at the worksites several times weekly to 
observe and guide them. 

At the three sites using the sheltered model, as in New York and Los 
Angeles, much of the assessment was informal. Cincinnati and St. Paul, 
however, administered some formal testing on an as-needed basis to 
supplement observation on such criteria as participants' stamina, mobility, 

coordination, reading and math skills, ability _o tell time, and vocational 
* 8 

interests. St. Paul even required formal assessments before placing 
participants into^ Phase 1 jobs. These tests were administered by outside 
-Agencies and, at an^ early^^poijnx.jLn^ the demonstration, caused delays for 
participants iTr~star^^^ program.^ In fact, the practical utility of 
£ oral SIT testing appeared to be limited. Cincinanti staff found that the 
tests tended to confirm their observations and, while helpful, were by no 
means essential. St. Paul staff reached a similar conclusion and ma'cle 
little use of the test information. 

The Phase 1 sheltered mo,del also raised questions about the utility of 
reimbursing participants at the minimum wage, rather than at the piece 
rates usuaJ ly paid in the workshops. The operators of the St. Paul site 
were opposed to this practice, believing it would generate resentment among 
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other workers at the facility. MDRC agreed to rescind the requirement at 
this site only, hoping it would also allow researchers to examine some of 
the implications of paying piece rates in Phase 1*^ In Cincinnati,* which 
paid the minimum wage, the different treatment of workers did cause some 
problems,, although riot to any great degree. Staff believed, however, that 
an important incentive was denied the program — the promise of a higher 
wage rate to motivate participants to move out of the workshop and into 
Phase. 2 jobs. 

One difficulty sheltered workshops face in providing high-quality 
training is a steady flo b w of work, which in these facilities is always 
dependent, on production contracts. If the c .tracts diminish, so do the 
work hours. In Cincinnati, during early 1982, business was poor, and the 
workshop was forced to reduce the work hours of all participants until 
production picked up later in the year. This also occurred in the Tucson 
sheltered workshop. 

Staff members in Cincinnati and Tucson identified an additional and 
potentially more serious problem in the u$e of a sheltered environment. 
Typically, the kinds of work available and the environment itself were 
different from those in Phase 2. Many staff believed that Phase 1 partici- 
pants were not being adequately prepared for work other than simple tasks, 
and that it was not even possible in the workshops to judge their suit- 
ability for more demanding jobs.** In an attempt to resolve this problem, 
Cincinnati increasingly moved the more capable Phase* 1 participants into 
temporary positions in its salvage warehouse or into janitorial services — 
positions thought to more closely resemble Phase 2 jobs. However, while 
sheltered workshop training can obviously be improved by providing jobs 
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that more, closely resemble regular ones, the nature of the contract .work 
performed in these settings makes it difficult to create such positions. 

Counterbalancing these factors, staff found a number of advantages to, 
the sheltered approach, the most important one being the opportunity for 
counselors to spend a good deal of time observing and assisting 
participants and getting to know their needs and abilities. Counselors 
were also better able to manipulate' the training environment to^-suit 
particular needs for example, varying the intensity of supervision or 
tailoring assignments' to certain tasks. The sheltered approach also 
reduced the program's job development burden, since staff did not have to 
pursue outside jobs for Phase 1 training. 

C. Other Phase L^Support Services 

While the degree to which staff members became directly involved in 
skills training varied among sites, counselors at all sites assumed direct 
responsibility for teaching participants appropriate work habits — 
although there were inter-site differences in the intensity and quality of 
these efforts. Generally, counselors discussed work habits with 
participants individually and as needed, either in special sessions or in 
the course of monitoring participants at work. Counselors also handled 
emotional and family problems in individual counseling; in fact, as one 
counselor put it, the staff tried to address "anything that might affect 
the ability to perform on the job." Sometimes, although not often, 
individual sessions included the parents of participants. 

Sites also conducted group counseling sessions, which were intended to 
provide peer support and to reinforce the kinds of behaviors and attitudes 
that staff members tried to inculcate in individual counseling. New York, 
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for example, ran a group counseling program of 16 weeks which included work 
orientation and required weakly attendance. The Cincinnati program ran 
weekly group meetings on work-related issues and held other workshops on 
special topics as judged necessary. In St. Paul, jdaily group sessions were 
held in approximately three-week cycles; Los Angeles and Tucson also had 
meetings on a regular basis. Counseling, both individual and group, also 
focused on independent living skills, such as how to manage money and 
handle living arrangements. Participants were taught how to budget their 
paychecks and given advice on topics such as proper social behavior, 
finding and managing an apartment, and shopping. 

Counselors also worked with participants on job search skills — how 
to look for a job, fill out an application, and act in an interview. These 
issues were usually addressed as participants approached the transition to 
Phase 2 jobs, many of which required interviews. At .most sites,, job search 
assistance was intensified in Phase 2 when competitive employment was 
closer at hand. 

Travel training, mostly in the form of teaching participants bus and 
subway routes to work, was an additional Phase 1 and 2 service, provided on 
an as-needed basis. This kind of training was most intensive in Tucson, 
where staff were ujore willing than at the other sites to enroll individuals 
who had only limited previous experience in independent travel. Roughly 
half of that site's participants needed to be taught travel routes between 
work, home and the STETS program, and how to use public transportation more 
generally. 
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III. Phase 2 Training and Support Services 

According to the program model, Phase 2 jobs were to approximate 
competitive jobs in their levels of responsibilities and skills. Moreover, 
the placements w.ere expected to be primarily in the private sector and to 
have the potential to evolve or "roll over 11 into competitive jobs, where 
the participants would become part of the regular workforce. At all sites, 
Phase 2 work experience occurred in regular rather than sheltered work 
settings. (See Table 3.1 for the distribution of types of jobs in Phase 
2.) " 

In New York and Los Angeles, where participants had already been 

placed in Phase 1 regular work settings, the Phase 2 training process 

continued as before. New York counselors continued to visit worksites. 

frequently and to use these visits for skills training and counseling. 

However, staff recognized that participants'' chances for succeeding in 

competitive* employment would be improved if they commanded a broader range 

of skills. As the program director explained: 

Because of the large unemployed labor pool, trainees' 
versatility is critically important. Firms want people they 
can shift across a variety of jobs* 

Another counselor explained: 

In Phase 2 you try to give them more diversity and work 
skills. If a person is doing photocopying, for example, you 
may try to teach them how to operate different types of 
copiers. 

Typically, counselors looked for new opportunities at the worksite and 
negotiated with the supervisor to make the desired change* Somje times a 
participant was shifted to a new job entirely, either at the same worksite 
or elsewhere. 

*j 
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Policies on Phase 2 at the Los Angeles Panorama City office resembled 

those in New York. According to the counselor in that office: 

I didn't do all that much actual changing of worksites for 
Phase 2. . But in some cases, participants were transferred 
to different departments where other disciplines could be 
worked on. In other cases, we'd just expand the job 
descriptions. If thfcre were new things that could be 
learned) and other responsibilities that could be given 
thefey we'd stretch the hours and responsibilities to test 
their capabilities. 

Participants supervised by the downtown Los Angeles office, however) 

often changed worksites (in both Phases 1 and 2) at the end of six weeks; 

the state program funding the majority of participants allowed no more than 

m 

250 hours of work experience in fully subsidized public and nonprofit 
agencies. Participants in that office also had few opportunities in which 
to expand their work skills. The pattern of low counselor contact seen in 
Phase 1 persisted into Phase 2, with the site conducting worksite visits 
only about once every three to four weeks. 

In Cincinnati, St. Paul and Tucson — where the shift from sheltered 
to non-sheltered work settings took place — the counselors' role during 
Phase 2 was similar to that adopted in New York. Counselors were 
substantially involved in the training process, emphasizing the importance 
of good work habits and providing general monitoring and support. 
Particularly during the first few weeks of a participant's Phase 2 
placement, all three of these sites provided what they referred to as 
"intensive training." Counseled spent several hours a day with 
participants, several days a week (and sometimes all day, every day of the 
week). They continued this type of training — although with gradually 
decreasing frequency — for up to several months . Once the counselor 
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judged a participant's performance satisfactory, visits diminished (but not 
below several times a month), arid monitoring depended more on counselor 
contact with the supervisor. 

Across all the sites, the level bf counselor involvement at the 
worksites depended. partly on the receptivity of employers. Some employers 
did not like counselors to conduct training at the place of business; it 
interfered with work, they said,* ^or the assistance was unnecessary 4 Sites 
in these cases talked to the supervisors by telephone and met with 
participants at the program offices. They also visited participants during 
lunch breaks or at other times not disruptive of work. As a New York 
counselor explained:' 



Most supervisors don't mind my visits. They like me. Some 
resent it if I take a trainee away from her work to talk to 
her. To ease this problem, I. try to see her when she's less 
busy. I feel better about pulling someone off the job in 
the public sector worksites. In the private sector, I 
sometimes have tc walk on eggshells. For example, I want 
John to get that job, so I'm more careful timing my 
visits. I try to adapt to their schedule. I may meet with 
the trainees on lunch breaks or before work. But some 
employers don't mind when I come. \ 



As the comment suggests, counselor interventions — both to shape the 
nature of jobs and to train and monitor — were easier when Phase 2 jobs 
were subsidized and in the public sector. Private employers who were 
paying the full costs of the participants' labor could be reluctant to 
involve program stsff. 

At all sites except New York, group counseling sessions were held less 
frequently during Phase 2 , largely because they were hard to schedule. 
Most participants worked in different locations and had varying hours. 
Limited public transporta *.on was an additional problem. In New York, where 
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there was easy access to public transportation and a much greater emphasis 
on public or nonprofit Phase 2 worksites, group sessions were more common. 
Especially towards the end of Phase 2, many participants attended special 
job search skills workshops. 

IV. Service Receipt in Phases 1 and 2 

As pointed out earlier, STETS guidelines stated that participants were 
to remain in Phase 1 and 2 no longer than 15 months. With soma cross-site 
variation, the program generally achieved this goal. As shown in Table 
3.2, participants on average , were enrolled in the program (including both 
phases) for 10.5 months. Eighty-eight percent were enrolled for 15 months 
or less. During their stay, participants worked an average of 743 hours 
in paid employment (Table 3.3), for an average of 2.7 employers. Average 
active time (i.e., the time during which participants were actively 
receiving program services) was 8.6 months (Table -A^l). 



The data also show that, as intended, Phase 1 work experience was 
limited. The average paid time of employment, at 414 hours, was below the 



loping Phase 2 jobs, Phase 1 employment took place over a longer time than 
originally anticipated. As is shown in Table 3.2, the average length of 
stay in that phase was 5.1 months, with almost three-quarters of the 
participle* staying in Phase 1 jobs, for over three months. Phase 2 length 
of stay was somewhat longer. Participants were enrolled in Phase 2 for an 
average of 6.3 months and worked an average of 470 paid hours (Table 3.3). 




500-hour limitation (Table 3.3). However,, because of difficulties in. deve- 



TABLE 3.2 



PERCENTAGE DISTRIBUTION AND AVERAGE DURATION OF ENROLLMENT 
IN STETS, BY PHASE AND PROGRAM SITE 



rnase Alia JJuration ot 


Lot 


New 


Cin- 






All 


tnroi lmenc 


Angeles 


York 


cinnati 


St. Paul 


Tucson 


Sites 


rntie i 














1 Month or Lett 


6.9 


6.2 


1.7 


11.4 


2.0 


5.4 


2-3 Months 


22 .A 


26.2 


25.0 


15.9 


11.8 


20.9 


4-5 Months 


46.6 


47.7 


25.0 


22.7 


25.5 


34.5 


6-7 Months 


13.8 


10.8 


23.3 


25.0 


25.5 


19.1 


8-9 Months 


5.2 


6.2 


11 .7 


9.1 


15.7 


9.4 


10 - 12 Months 


3. A 


1 .5 


6.7 


6 .8 


11 .8 


5.8 


More Than 12 Months 


1.7 


1.5 


6.7 


9.1 


7.8 


5.0 


Ta *• m 1 

XO ual 


1 AA A 

100 .0 


1 AA A 

100 .0 


1 00 .0 


100 .0 


100.0 ■ 


100.0 


Average miration vnonLiisj 


4.1 


3 .8 


5.5 


5.7 


6.6 


5.1 


wumDer or enrol lees 


58 


65 


60 


44 


51 


278 


r n a s e z 














1 Month or Less 


2.8 


« 7.7 


2.5 


4.5 


5.6 


4.8 


2-3 Months 


11.1 


15.4 


1 0.0 


36.4 


11.1 


15.1 


4-5 Months 


11.1 ■ 


21.2 


27.5 


27.3 


25.0 


22.0 


6-7 Months 


13.9, 


13.5 


25.0 


4.5 


19.4 


16.1 


8 # - 9 Months 


13.9/ 


19.2 


25,0 


27.3 


19.4 


20.4 


10 - 12 Months 


44.4 


17.3 


10.0 


0.0 


8.3 


17.2 


More Than 12 Months 


2.8 


• 5.8 


0.0 


0.0 


11.1 


4.3 


10 cai 


1 AA A 
100 rO 


100 .0 


100 .0 


100 .0 


100 .0 


100.0 


Average i/uracion vMontns/ 


7 .6 


6 .5 


" 5.7 


4.2 


6 .5 


6.3. 


wumDer or tnroiiees 


36 


52 


AO 


22 


36 


186 


rnases i ana z voomDineo/ 








• 




- 


1 Month or Less 


3. A 


1.5 


1.6 


2.2 


0.0 


1 .8 


2-3 Months 


1.7 


4.6 


" 6.5 


8.7 


7.5 


5.6 


A - 5 Months 


15.5 


7.7 


12.9 


4.3 


3.8 


9.2 


6 r 7 Months 


13.7 


10.8 


12.9 


21.7 


11.3 


13.7 


8 - 9 Months 


10.3 


18.5 


8.1 


15.2 


11,3 


12.7 


10-12 Months 


8.6 


23.1 


12.9 


17.4 


7.5 


14.1 


13 - 15 Months 


46.6 


26.2 


41 .9 


15.2 


20.8 


31.0 


More Than 15 Months 


0.0 


7.7 


3.2 


15.2 


37.7 


12.0 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


Average Duration (Months) 


10.1 


10.2 


9.9 


9.6 


12.8 


10.5 


Number of Enrol lees 


58 


65 


62 


46 


53 


284 



(continued) 
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TABLE 3.2 (continued) 



\ 



SOURCE: MDRC tabulations of Monthly Statu* Change Forms in the STETS 
Intonation System. , 



NOTES : 
rounding. 



Percentage distributions may not add exactly to 100.0 because of 

* ■ . 

a ' 

Includes Phase 1 'enrollment time for enrollees who entered Phase 

Includes Phase 2 enrollment time for enrol lees who entered*(Phase 
c 1 

Excludes one enrol lee in Los AngeTffir f or whom data are missing. 

includes Phase 1 and Phase 2 enrollment, time for persons who ever 
enrolled in STETS. „ • 



TABLE 3.3 

DURATION OF PAID -EMPLOYMENT, 
BY PHASE AND PROGRAM SITE 



Phase and Duration of 
Paid Employment 


Lo s 
Angeles 


New 
York 


..cinnati 


St. Paul 


Tucson 


All 

Sites 


Phase 1* 














Average Hours Worked 
Average Days Worked 


301.3 
42 .A 


438.6- 
65.0 


407 .6' 
79.2 


502.9 
• 77.0 ■ 


425.6 
• 98.3" 


414.0 
72.1 


Number of Participants 


49 


65 


60. 


42 . 


49 


265 b 


Phase 2° 




• • 










Average Hours Worked 
Average Days Worked 


736.0 
96.5 


391 .1 
63.6 


458.8 
77.6 


376.7 . 
77.4 


381 .4 
73.1 


470.3 
76\6 


-w, Number of Participants 


37 


52 


40 


22 


36 


187 


Phase 1 & 2 (Combined ) d 














Average Hours Worked 
Average Days Worked 


857 .0 
U5.3* 


751 .4 
115.9 


7 13 .5 
130.9 


700.2 
117.5 


691 .7 
149 .0 


743 .0 
125.6 


Number of Participants 


49 


65 


60 


42 


50 


266 



SOURCE: MDRC tabulations of Monthly Activity Forms in the STETS- 
Information System. . - * . 

* a 

NOTES: Includes Phase 1 employment for enrollees -assigned to a Phase 1 

job. 

b Excludes one participant in, Tucson for whom data are missing. 

c 

Includes Phase 2 employment for enrollees assigned to a Phase 2 

job. 

d 

Includes Phase 1 and Phase 2 employment for enrollees assigned 
to at least *a ^hase-JL job. 
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V. Phase 3 Services ^ 0 

As originally conceived, the third and final STETS stage was to begin 
when participants" wages were no longer subsidized by the program. During 
Phase 3, staff were to continue their efforts to solve problems and to help 
participants adjust to unsubsidized employment. Phase 3 was supposed to 
last for no longer than six months. 

In practice, hpwever, the distinction between Phases 2 and 3 became 
blurred, probably because in all sites, except New York, many Phase 2 jobs 
were unsubsidized. This was, of course, a positive development because it 
reduced program operating costs, but it also raised questions about the 
structure of "Phase 3. When the pattern iirst became evident, MDRC sought, 
to define this period as one in which: 1) the program yas no longer provid- 
ing "substantial 11 support services to the participant, and 2) the partici- 
pant, staff and employer agreed that the participant was not a trainee, but 
a permanent member of the regular workforce. ^ > 

Nevertheless, the distinctions between Phases 2 and 3 were never 
precise. Regular Phase 2 worksite visits continued into Phase 3 in all 
sites except Los Angeles, although they generally occurred less frequently. 
In some cases, they even increased, at lea§t temporarily, because regular 
employment presented new problems to participants. In St. Paul, for 
example, staff observed that, within three months of entering Phase 3, many 
participants moved into independent v living situations, which in turn 
created^s tress that could affect work performance. 

The New York director described the more typical set of difficulties 
confronting participants whose Phase 3 jobs* were in new firms, rather than 
in firms where they had worked in Phase 2 jobs: 



Newly hired (Phase 3) STETS participants are often in a 
vulnerable position^ employers are increasingly likely to 
fire any employee who is having some difficulty when there is 
a large pool of applicants available to take his or her 
place. For example, one STETS participant was learning her 
job at a slower pace than other workers hired at th6 same 
time. The supervisor was considering termination. However, 
the follow-up counselor began working alongside her on a 
daily basis and the participant was able to meet performance 
standards. , 

Such temporary increases in support notwithstanding, staff contact 
with participants declined considerably over time. As Table 3.4 shows, 
fewer participants were in contact with staff members at the end of Phase 3 
than in the beginning, and the number of contacts had also decreased. 

It is important to note, however, that participant contact did not 
necessarily end after six months. In the New York program, for example, 
counselors tried to keep in touch with participants at least bimonthly for 
an additional six months. In fact, most staff members believed that it was 
necessary to have a longer post-program contact period than that originally 
delineated in the STETS model. 
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TABLE 3 .4 



INTENSITY OF PROGRAM CONTACT WITH PHAS.E 3 PARTICIPANTS, 
BY MONTH IN PHASE 3 AND PROGRAM SITE 
THROUGH DECEMBER 1983 a 



Month in Phase 3 and 
Measure of Intensity 


Los 
Angeles 


New 
York 


Cin- 
cinnati 


St. Paul 


Tucson 


All 
Sites 


First Month 














Average Number of Contacts 
Percent of Participants 
. Contacted 


0.8 
78.9 


1 .8 
65.6 


2.9 
100 .0 


1 .9 
72.7 


3.4 
93.1 


2.3 
82 ..6 


Number of Participants in 
Phase 3 at Least One Month 


19 


32 


24 


11 


29 


115 


Third Month 














Average Number of Contacts 
Percent of Participants 
Contacted 


0.7 
57.9 


0.9 
62.5 


2.1 
90.9 


2.8 
80 .0 


2.8 
82. 8 


1.8 
74.1 


Number of Participants in 
Phase 3 at Least Three Months 


"19 


32 


22 


10 


29 


112 


Sixth Month 














Average Number of Contacts 
Percent of Participants 
Contacted 


0.6 
54.5 


0 .5 
40 .0 


1 .4 
90.5 


1 .2 

44.4 


2.4 
72.7 


1.2 

61.3 * 


Number of Participants in 
Phase 3 at Least Six Months 


.11 


30 


21 


9 


22 


93 



SOURCE: MDRC tabulations of Monthly Follow-up Forms i .he STETS Information 
System. 

NOTES: Because follow-up data were collected only through December 1983, 

contacts with participants still in -Phase 3 during early 1984 are not included in 
these calculations. *~ 





CHAPTER IV 



JOB DEVELOPMENT 



The rate at which participants could be moved through the first two 
program phases and into permanent jabs, and the quality of the match 
between thei<- skills and employers'' expectations, were both largely 



dependent on the effectiveness of job development. For several reasons, 
planners had anticipated that this task would pose some difficulties. 
There was an impending recession .and a relatively lairge number of 
enrollees, and most sites had only limited experience in private sector job 



negative attitudes toward hiring the .mentally retarded. 

This chapter examines how well the sites were able to overcome these 



]?hase 1 and 2 training positions (in non-sheltered settings) and for 
regular Phase 3 jobs. It also focuses on the strategies and results of 
each site. 

I. Job Development Strategies 

At all sites — with the exception of Los Angeles and (during 19613) 
St. Paul — job development was the sole responsibility of one or more 
staff members. (In Los Angeles, consistent with the practice of the spon- 
soring agency, each counselor was expected both to provide training assist- 
ance and find jobs for the individuals on his or her caseload.) Job devel- 
opers on the whole had varied backgrounds. Some Came from the business 
world, while others had . counseling or other human services backgrounds. 



development . 



Past research also suggested that many employers hold 



problems. 



It describes the program's job development efforts for both 
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Few, however, had previous experience in developing jobs for mentally 
retarded persons. Information on how staff members developed jobs was 
obtained in interviews with MDRC researchers and from an eight-month job 
development contact study, which MDRC conducted in four of the 
demonstration sites. The Los Angeles program was excluded from this last 
analysis because of data collection problems, (See Appendix B.) 

Job developers identified worksites by scanning newspaper want ads and 
telephone directories, soliciting referrals from friends and professional 
acquaintances, and simply spotting local firms ^and agencies In the communi- 
ty. More often than not, they had to rely on "cold contacts" with employ- 
ers. This was less true in New York and Los Angeles, where job developers 
were able to rely on a network of employers in ^public and nonprofit agen- 
cies where the STETS sponsoring agency had previously placed other clients. 

From September 1, 1982 through April 30, 1983, job developers kept a 

record — using standardized MDRC forms — of the contacts they made with 

employers from whom tjiey hoped to obtain training slots or Phase 3 jobs. 

For study purposes, a distinction was made between ,a "preliminary contact" 

(the initial attempt to reach the appropriate person) and a "substantive 

contact," wherein the job developer had an opportunity to describe and 
« 

"sell" the program. For each substantive contact, job developers were 
asked to specify the $ource from which they idenuified the employer. 

The results in Table 4.1 show that, among the 621 * employers who 
granted at least one substantive contact (most were in not-for-profit 
firms), almost 80 percent had come initially from cold contacts, a finding 
consistent with the reports from staff interviews. Only about 4 percent 

» 
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TABLE 4.1 



PERCENTAGE DISTRIBUTION OF JOB DEVELOPERS' CONTACT SOURCES FOR EMPLOYERS 

WHO GRANTED A SUBSTANTIVE CONTACT 



Source of Contact 


New 

York 


Cin- 
cinnati 


St. Paul 


Tucsjn 


■ 

All 

Sites 


Cold Contact 


AA 7 
HO • / 


58 .8 


92 .9 


100.0 


79.7 


Previous Placements by 
STETS Sponsoring Agency 


12.6 


2.9 


0.6 


0.0 


3.4 


Personal/Professional 
Acquaintance 


1 A 1 
10 .J 


0 .0 


1 .2 


0.0 


3.9 


Contact Made Prior to 
STETS Contact Study 


8.2 


31 .4 


0 .0 


0.0 


6.9 


Other 


16.3 


6.9 


5.3 


0.0 


•6.1 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


Number of Employers 


135 


102 


169 


215 


621 8 


SOURCE: Job Development Contact Study. 








NOTES: Percentage 
b&cause of rounding. 


distribut 


ions may not 


add exactly 


to 100.0 



- Los Angeles has been excluded from this analysis due to 
problems m data collection. 

1 

f • c « Excludes four employers in New York, four in Cincinnati, 
four in St. Paul and two m Tucson for whom data are missing. 



\ 




do 
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resulted from the job developer's own personal or professional relation- 
ships, and a similar amount came from friends and acquaintances. In about 
7 percent of the cases, job developers had been in contact with the 
employers before the beginning of the study. Three percent of the jobs 
were in firms where the sponsoring agency had previously placed clients. 

Table 4.1 also reveals some inter-site differences. For example, New 
York and Cincinnati used sources other than cold contacts more often than 
the other sites. Probably because Job Path had been in operation for two 
years before STETS, New York job developers had a larger number of personal 
or professional contacts. Staff in those two sites also concentrated on 
job development at an early point in the demonstration, and had already 
contacted a good number of employers by the time the study began. 

At all sites, developers looked for jobs in firms of all sizes and in 
many kinds of industries. They soon discovered, however, that the level of 
effort necessary for pursuing jobs in large firms was considerably greater 
than that for small ones because of the difficulty in reaching persons with 
hiring authority. Thus, while they continued to keep in touch with the 
large firms, their efforts centered on small businesses and came to focus 
on the rather narrow range of jobs that had characterized the program 
almost from the outset. Typically, the jobs they developed included jani- 
torial and maintenance positions, dishwashing and food service jobs, and 
housekeeping and other service jobsl (In New York, there was an emphasis 
on mailroom messenger and stock clerk positions.) Job developers reported 
that the competence range and skill levels of most participants were best 
suited for these jobs. Additionally, ^he high rate of turnover in these 
positions was a strong selling point for STETS workers. 
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Onco potential worksites were identified, job developers called employ- 
ers and/or visited them in person to encourage them to become involved in 
the program. Employers who were willing to listen were given a sales 
presentation that was fairly consistent across sites , emphasizing that 
hiring STETS participants made good business sense. Employers were told, 
for example, that participants were capable of performing many jobs well, 
and because their opportunities were limited, they were not likely to leave 
low-skilled positions quickly. A New York job developer repeated for an 
MDRC researcher part of her typical presentation, which focused on the 
economic benefits of reliable employees: 

Where are your needs? Do you have a high turnover rate in 
any area? Do you have any areas where people get bored? Our 
people are good workers. They just happen to be handicapped. 
When people come to you looking for a job, I'm not saying 
hire them just because they are handicapped and it's 
affirmative action, but because it will be good for your 
needs. We're providing you with a mailroom worker, for 
example, who can be a good worker, who will show up every 
day. 

Because of possible employer misconceptions about retarded persons and 

their abilities, participants were typically described as "slow learners," 

rather than mentally retarded. When that term came up, job developers 

distinguished it from mental illness, and emphasized that it was mild. As 

a New York job developer explained: 

I will say that we work with developmentally disabled young 
adults. I never say mentally retarded, I say mildly retard- 
ed. If I say mentally, it gives the wrong impression. ,0ur 
clients aren't mental. They don't have emotional disorders. 
They're not in mental hospitals. 

In the New York program, the title "job developer" was eventually 
changed to "account executive" to further highlight the program's orienta- 
tion toward the business world. As the director of job development at that 
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site noted, "We're trying to shed our social service skin. 11 Similarly, 
Cincinnati developers were caD.ed "employment specialists," 

For Phase 3 positions, employers were told that STETS participants 
were, in, effect, "prescreened" workers (i.e. , they had program work 
experience) and that it was less risky to hire them than to hire other 
persons "off the street." For Phase 2 positions, employers were told that 
some participants might not yet be good workers. However, they were 
assured that they would not face the prospect of having to "fire" handi- 
capped workers; the program would assume the burden of removing poor per- 
formers from the job. Another incentive stressed by the job developers 
was program assistance in training STETS participants for a specific job. 

Employers were also offered financial incentives . As discussed 
earlier, in New York and Los Angeles full wage subsidies were available for 
work experience in non-sheltered Phase 1 positions in public and nonprofit 
agencies. Usually these subsidies were paid by CETA funds, which allowed 
up to 1,000 hours of work experience which could extend into Phase 2 (as 
long as the work was performed in these agencies). During the period of 
CETA funding, participants remained on the STETS payroll and received 
fringe benefits. The Los Angeles site, as mentioned earlier, was also 
assisted by a state youth program. 

New York extended non-CETA-funded wage subsidies to some private 
sector firms in the form of "service contracts," which lasted from 2 to 12 
weeks. In. these agreements, as with other subsidies, the participant 
stayed on the program's payroll, and the program was responsible for fringe 
benefits (and in some cases for part of the wage). The employer paid the 
program a "fee" covering the part of the wage previously agreed to. 
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Up to the last year in St. Paul, Phase 2 wages for public and private 
sector jobs could be subsidized for up to eight weeks by on-the-job 
training funds from a national program of the Association for Retarded 
Citizens (ARC). Since the participant was required to be on the employer's 
payroll, with that employer reimbursed directly by ARC, the STETS job 
developers offered to assist employers in completing the paperwork. To a 
lesser degree, the other sites also used subsidies. In Cincinnati, Phase 2 
employers were frequently offered a 25 percent subsidy, which lasted an 
average of 12 weeks and was paid out of the site's general STETS budget. 

Panel A in Table 4.2 presents MIS data on the percentage of 
participants on an employer's payroll in their first Phase 2 jobs. (In 
Phase 1, no participaxits were on the employer's payroll.) The table 
indicates that, in all sites except New York, the proportion included at 
least half of the Phase 2 .participants . The proportion is especially high 
in St- Paul and Tucson, where 100 percent and 89 percent of participants, 
respectively, vere on employers' payrolls. In New York, in contrast, only 
4 percent of Phase 2. participants were so listed. This reflects that 
site's emph; is on the use of CETA-funded Phase 2 jobs, many of which were 
extensions of Phase 1 positions. 

Fanel B in Table » .2 shows the proportion of the wage paid by „ s ie 
employer, whether or not a participant was on the employer's payroll in his 
or her first Phase 2 job. Across the demonstration, employers paid at 
least part of the wage for over 57 percent of Phase 2 participants, and 
they paid the entire wage in 37 percent of the cases. Full payment varied 
considerably across the sites, however: from a low of 6 percent in New York 
to a high of 76 percent in St. Paul. 
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TABLE 4.2 



A.. PERCENTAGE DISTRIBUTION OF PHASE 2 PARTICIPANTS WHO WERE ON THE 
EMPLOYER'S PAYROLL IN THEIR FIRST PHASE 2 JOB, BY PROGRAM SITE 



Payroll Status 


Los 
Angeles 


New 
York 


Cin- 
cinnati 


St. Paul 


Tucson 


All 
Sites 


On Payroll 


50.0 


3.8 


67.5 


100.0 


88.9 


54.1 


Not On Payroll 


44.4 


92.3 


32.5 


0 .0 


8.3 


. 43.2 


Don't Know 


5.6 


3.8 


0.0 


0.0 


2.8 


2.7 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


100 .0 


Number of Participants 


36 


52 


40 


21 


36 


185 3 



B. PERCENTAGE DISTRIBUTION OF THE PROPORTION OF WAGE PAID BY THE EMPLOYER 
FOR PHASE 2 PARTICIPANTS IN THEIR FIRST PHASE 2 JOB, BY PROGRAM SITE 

. i 



Percent of Wage 
Paid By Employer 


Los 
Angeles 


New 
York 


Cin- 
cinnati 


St.. Paul 


Tucson 


All 

Sites 


0 


47 .2 


* 

63.5 


30.0 


14.3 


38.9 


42.7 


1-50 


19.4 


28.9 


15.0 


9.5 


8.3 


17.8 


51 - 99 


0.0 


1.9 


10.0 


0.0 


0.0 


2.7 


100 


33.3 


5.8 


45.0 


76.2 


52.8 


36.8 


Total ' 


100.0 


100 .0 


100 .0 


100.0 


100.0 


100..0 


Number of Participants 


36 


52 


40 


21 


36 


185 3 



SOURCE: MDRC tabulations of Monthly Activity Forms in the STETS 
Information System, 

NOTES: Percentage distributions may not add exactly to 100.0 because 
of rounding. 

Excludes one participant in Los Angeles and one participant 
in St. Paul for whom data are missing. 
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Planners for STETS did not anticipate a relatively large group of 
employers who woufa be willing to accept Phase 2 participants without any 
wage subsidies. The findings suggest, however, that it is feasible to make 
such arrangements with a sizable number of employers. Future STETS 
programs should take this possibility into account in developing Phase 2 
jobs. 

An additional incentive to employers was the Targeted Jobs Tax Credit, 
which private businesses were eligible to receive for each STETS 
participant they hired on a permanent basis (Phase 3). This credit 
amounted to a 50 percent reimbursement of first-year wages up to $6,000 and 
25 percent of second-year wages to that same level. Job developers again 
offered to help employers complete the necessary paperwork. 

In all sites except Los Angeles, job developers looked for positions 
on a continuous basis and tried to find jobs appropriate for a wide range 
of participants. In Los Angeles, however, job development was an indivi- 
dually tailored process. Counselors looked for jobs for specific clients 
when the need arose, and did not make ongoing efforts to develop a reserve. 

In several sites > the directors of the program or its sponsoring 
agency assisted the job developers in making initial contacts, particularly 
in larger companies. In most cases, they tried to elicit the commitment of 
high-level managers who, it was hoped, might make it easier to place parti- 
cipants into the^firms. Such contacts were made chiefly in individual meet- 
ings, luncheons with representatives of local businesses, and through 
presentations made to civic organizations, such as the Rotary Club. 

Systematic involvement of higher-level staff or the sponsoring agency 
was most common at the New York site, where both the project director and 
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the dirtctor of job development aide frequent presentation! to business 
groups. As part of that effort, they used a slide presentation of STETS 
participants at work. According to the director of job development, the 
approach worked well: 

A lot of people, We are understanding now, soe retarded 
people in the worst possible light — they are Mongoloid, 
they drool, etc. So we take pictures of our clients on the 
job so they can see that retarded people look like everyone 
else. This makes people feel more comfortable in tents of 
working with retarded people. They see thoa working with 
other workers and fitting in. People say, "They diin't look 
any different. It's not what I imagined. " The slide 
presentation has been effective. 

The New York site created a Busines; /Labor Advisory Council, which 
included business leaders, company personnel directors and representatives 
from l'bor unions. The advieory council was particularly helpful in defin- 
ing the jobs and industries that showed growth potential and that aight be 
appropriate for STETS participants. Soae council members directed job 
developers to specific firas, and others conducted mock interviews with 
selected trainees who were preparing to enter unsubsidized jobs. The 
Cincinnati site also set up a job advisory council, which was comprised of 
leaders of about 10 area businesses to identify the kinds of jobs and 
industries on which the program should focus its job development efforts. 
Tucson, too, had a small advisory council late in the demonstration, but 
St. Faul end Los Angeles chose not to use one. 

As a final step in the process, job developers followed up on 
employers who sounded genuinely interested, but who had no mediate open- 
ings. They usually tried to maintain contact with these employers by tele- 
phoning periodically. The Cincinnati site also sent copies of a newsletter 
on the program to potential sponsors. 
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II. Job Development Result* 

Results of the job development efforts were mixed. Sites eventually 
found both training positions and Phase 3 jobs for almost all participants 
who appeared willing to accept the positions and were capable of meeting 
this demands. In a few cases, participants were terminated from the program 
simply because no suitable Phase 3 jobs could be found. Occasionally, 
participants refused the specific jobs the program was able to find. 

In general, job development was a difficult challenge for all sites, 
but two sites had particular problems. The Phase 3 placement rates (see 
Chapter V), interviews with staff member s 9 and the job development contact 
study all suggest that the problems were most severe in St. Paul and Los 
Angeles. 

The low acceptance rate of employers is apparent in the contact 
study. During the study period, only 4 percent of the 1,027 employers ever 
contacted across the demonstration (excluding Los Angeles) agreed to 
provide training positions or Phase 3 jobs for participants. St Paul had 
the lowest agreement rate (2 percent) while Tucson and Cincinnati had the 
highest (5 percent). New York achieved an agreement rate of 4 percent. It 
should be noted that these outcomes are not surprising; in fact, they are 
similar to those achieved by other employment programs, including those 
serving non-retarded persons. For example, in a special study of, employers 
in the Youth Incentive Entitlement Pilot Projects (a project for low-income 
youths), only 13 percent of a sample of private sector employers in Detroit 
and Baltimore who were offered either a 50, 75 or 100 percent wage subsidy 
agreed to take on a program participant. Among those offered the 50 
percent subsidy, the agreement rate was only 5 percent (Ball et al. , 1981). 
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For employers who granted the STET8 program a substantive contact but 
did not take on a participant, the reason given in over three-fourths of 
the cases vas no appropriate openings. Only 3 percent said they were 
concerned about hiring retarded persons. Job developers believed, however, 
that these results understated the actual level of concern. 

The fact that the overall supply, of jobs available to the program vas 
tight had important implications. First, sites could not be as selective 
as they had hoped in matching participants to jobs. Second, there were 
delays in moving individuals into Phase 2 jobs after their allotted Phase 1 
hours vere completed, and similar delays between Phases 2 and 3. As a 
result, many youths vere in an inactive or "hold 1 * status for weeks, or even 
months. 1 The delays vere most severe in Los Angeles and St. Paul, but 
occurred to some extent in all sites. 

lit. Obstacles to Job Development 

Job development vas constrained by a variety of problems, one of the 
most important of vhich vas the veak labor market. When the demonstration 
began, national unemployment vas on an upward path, exceeding an average 
annual rate of 9 percent in 1982 and 1983. (See Table 4.3.) Moreover, the 
unemployment rate for youths having similar characteristics to STETS 
enrol lees (20 years old, on average, and 45 percent non-white) vas 
considerably higher. 

Because many companies had imposed hiring freezes or had promised jobs 
to workers vhp had been laid off, the program had to compete with other job 
jeekers for fewer jobs. Another effect, according to staff* t#as a reduced 
rate of turnover in many low-skilled, entry-level jobs; many workers 
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TABLE 4.3 

UNEMPLOYMENT BATES, BY SITE AND YEAR OF DEMONSTRATION 



Year " 


Lot 
Ante let 


New 

York 


Cin- 
ciontti 


St. Ptul 


Tuctoo 


United 
States 


1981 a 


7.3 


8.8 


9.7 


4.5 


5.6 


8.4 


1982 


9.3 


9.5 


10.4 b 


6.5 


9.4 


9.7 


1983 


9.7 


9.4 


9.5 C 


6.6 


9.2 


9.5 


1981 - 1983 d ,• 


9.3 


9.4 


10.0 


6.3 


8.9 


' 9.5 



SOURCE: Employment 4 Earnings. U.S. Departpent of Labor, 
Bureau of Labor Statistics. 

Aret definition! tre as follows: Lot Angeles/Long 
Beach (Long Angeles County); New York City (5 Boroughs); Cincinnati 
(Clermont, Hamilton 6 Warren Counties, Ohio; Boone, Campbell 4 Kenton 
Counties, Kentucky; Dearborn County, Indiana); St. Paul (Anoka, 
Carver, Chisago, Dakota, Hennepin, Ramsey, Scott, Washington 4 Wright 
Counties); Tucson (Pima County) . 

NOTES: *Data for 1981 are from October through December only. 
Participant intake for STETS did not begin until October 1981 . 

b The Bureau of Labor Statistics provided data only for 
March-July, September and October in 1982 for Cincinnati. 

The Bureau of Labor Statistics provided data only, for 
May, July. August. October and November in 1983 for Cincinnati. 

Shis row is a weighted average unemployment rate for 
the period October 1981 through December 1983 using monthly data. 
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capable of jobs requiring greater skills were more willing to remain in 
low-skilled positions. Some jobs were consolidated, and the variety of 
tasks in entry-level positions sometimes increased responsibilities to a 
point where the jobs exceeded the capabilities of most STPtS participants. 

Factors other than the state of the economy also played a major part 
in affecting outcomes. The way in .which sites handled job development was 
especially important. For example, St. Paul, the site with the lowest 
unemployment rate through the end of 1982, changed job developers several 
times. During 1983, the position was discontinued, and its responsi- 
bilities were divided among project directors and counselors. This move 
may well have impaired that site's effectiveness in securing jobs. 

In Los Angeles, the obstacle seems to have been the site's individual- 
ised approach. Searching for jobs on an as-needed basis, rather than 
trying to develop a reserve, was not an efficient practice. Furthermore, 
the merging of the job development and training responsibilities made it 
difficult for counselors to perform either function well. 

In several sites, job development was hampered by the fact that parti- 
cipants lived in large cities, many of which had limited public transporta- 
tion. This problem, particularly acute in Los Angeles, narrowed the job 
possibilities, since most participants (virtually none of whom could drive) 
had to be able to reach work by public transportation. 

Another important constraint was the short period of the demonstra- 
tion. As noted previously, site operations, originally planned for a 
three-year period, were truncated by funding limitations. One result was 
that job developers, who had little experience in locating work for 
mentally retarded persons, had insufficient time in which to learn 
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effective techniques, particularly since job development started late in 
•pat sites* It also took time to build a positive reputation in the cob- 
■unities: For example, job developers had few "success stories" to substan- 
tiate their claims about participant performance because , as the johs were 
being developed, the majority of participants had not yet entered Phase 3 f 
and few had completed it. On this point, it is instructive to recall New 
York's experience* This site's effort was made much easier because the 
parent organization ran a similar program with a positive track record. A 
network of previous employers could provide letters of recommendation and 
other more informal endorsements which staff could give to prospective 
employers. 

TV., Employers' Reasons for Extend in£ Job Opportunities to STETS 
Participants 

Obstacles notwithstanding, many employers did provide jobs. This 
section, which primarily relies on data from the employer survey, presents 
some of the reasons why employers of Phase 2 participants agreed to offer 
training slots, some of which later became regular jobs. 2 

In the survey, a sample of employers who had accepted one or more 
Phase 2 participants was asked a series of questions about that decision. 
One question was open-ended: "When you were first asked to bring on a STETS 
participant, what were your reasons for deciding to do so — aside from 
reasons you would normally have for bringing on a new employee? 11 The 
responses are presented in Table 4.4. The first column shows that by far 
the most commonly mentioned reason was altruism: 78 percent of the 
employers said that a desire to help a participant or the community was a 
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*iA»LE 4.4 

REASONS CITED IT EMPLOTEES FOt ACCEPT I »C A PIAII 2 PAATICIPAJTT, 

IT CECTOE 





Pircaat of Enployin 


Stctor aad Rtaioa 


IJtftA 
WW 

Citad 
Rt aioc 


Vao Cittd Riaioa 
ci Only or Ho it 
Inportaat Riaion 


All Sictori 






Dctirt to hilp partlcipaat/coannuaity ' 


77.1 


53 .2 


rrograa attittaaci vicb participant 
conpaatatioa 


i a 9 
ia *2 


12 .1 


Traiaiag attiitaaca offtrtd by 
Program 


17.2 


0.4 


Prtviout aaployattut ixptritaci vita 
baadicappad piopli 


15 .2 


a *% 


Mad jobi appropriatt for handicappad 


14.1 


4.3 


Pravioui non-nap loymant tapir iaaci 
with baadicappad paopla 


11.1 


1 .1 


Praiantatioo by STETS itaff 


11.1 


2.1 


Ealativi or friind it baadicappad 


7.1 


2.1 


Nuaibar of Eaploytri 


99 


f4 b 


Pot—Profit Sictor 






Tax Cradit c 


21.1 


10.5 


Participaati bad prior training ia 
STETS 0 


7.0 


0.0 


Huaibar of Esployiri 


57 


57 



SOURCE: KDEC tabulation! of data f roar anployar riipomai to tba 
STETS Enploytr Surviy Initrumant. 

ROTES: Tha innriy vai aaainiittrtd to anploytri of Phaia 2 
participant! at all fiva STETS litu bttvtto Dtctnbtr 1982 and April 1983 . 

*So»t aaployiri citad sort taan ona riaion. 

^Excludas 5 anploytri who citad sore than ona raaion for 
accapting a STETS participant but did not citt any ona raaion ai baing tba 
moit important. 

c 0nly for-profit first vara tligiblt for tbi tax credit. 

"Public iictor and non-profit firni art tscindad from tba 
bait for tbii rupoait btcaun ia thi Ktw Tork and Loi Aagilai litu 
auatroui Phait. 2 jobi ia thtit nctort ware origiaally Pbaia 1 jobi. Ia 
■acb circuantancai, Phait 2 participaati would not havt bad prior traiaing 
in STETS at tht tin* of tht amployir'i daeiiioa, linci they vara initially 
accaptad by tba anployar ai Phait 1 traiaaai. 
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factor in their hiring decision. . When respondents mentioned more than one 
reason, they were asked to specify the most important one* The second 
column in the table shows the proportion of employers to cite a specific 
response as the only or the most important reason. Altruism again is seen 
as the dominant factor: Over half of the employers gave it priority. 

To some extent, the frequency with which altruism was mentioned may 
reflect a response bias in the survey; some employers may have wanted to 
appear socially concerned. Nevertheless, the reaction is not surprising: 
Altruism is a reasonable motive for wanting to hire handicapped persons, 
especially those in a special employment program. Although job developers 
emphasized enrol lees" competencies, many employers may have felt that STETS 
participants were a group deserving of special attention. 

It should be emphasized, however, that almost 47 percent of the employ- 
ers did not cite altruism as the only or the most important reason for 
sponsoring a participant. Clearly, other considerations were important. To 
examine them more closely, employers were asked additional questions about 
specific incentives job developers offered: wage subsidies, program 
assistance in training and monitoring participants, the Targeted Jobs Tax 
Credit, and participants' prior work experience in STETS (Phase 1). 
Employers were asked to rate the relative importance of each incentive on 
a scale ranging from 'Very important" to "not important At all." 

The results, presented in Table 4.5, show that, among all employers 
iuteviewed, 43 percent rated the financial subsidy as very or somewhat 
important. Three-quarters gave a similar rating to the program's offer of 
assistance in providing on-the-job training and monitoring of participants. 
Among employers in for-profit firms who could receive a tax credit, 42 
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TABLE 4.5 



PERCENTAGE DISTRIBUTION OF EMPLOYER ASSESSMENTS 

OF THE IMPORTANCE OF SELECTED INCENTIVES 
FOR ACCEPTING A PHASE 2 PARTICIPANT, BY SECTOR 



Sector and Incentive 



Little or Don't 
Very Somewhat Not At All Know 



Total 



All Sectors 

Financial Subsidy 

Training Assistance 
Provided by gram 




100.0 (99) 



100.0 (99) 



For-Profit Sector 
Tax Credit b 



Participant Had Pjior 
Training in S'xETS 



100.0 (57) 



100.0 (99) 



SOURCE: MDRC tabulations of data from employer responses to the STETS 
Employer Survey Instrument. 

NOTES: The survey was administered to employers of Phase 2 participants 
at all five STETS siter between December 1982 and April 1983 • 

Numbers in parentheses refer to the sample site. 

Percentage distributions nay not add exactly to 100.0 because of 

rounding. 

'includes 52 employers who accepted a Phase 2 STETS participant 
vith no financial subsidy and 4 employers who received compensation out said 
that it was of little or no importance in their decision. 

b 

Only for-profit firms were eligible for the tax credit. 

c These employers indicated that they were not aware that they 
might be eligible to receive a tax credit for hiring a STETS participant. 

^Public sector and non-profit firms are excluded from the base for 
this response because in the New York and Los Angeles sites numerous Phase 2 
jobs in these sectors were originally Phase 1 jobs. In such circumstances, 
Phase 2 participants would not have had prior training in STETS at the time of 
the employer's decision, since they were initially accepted by the employer as 
Phase 1 trainees. 
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percent rated this credit at very or somewhat important* In addition, 58 

percent of these firms considered prior training in STETS as somewhat or 

4 t 

very important. 

It is also notable that not all incentives were important to all 
employers. For example, over half indicated that wage subsidies were not 
very important; indeed, many accepted participants without any subsidies. 
Yet it is clear that, without some incentives, job development could have 
been considerably more difficult than it was. 

Finally, it should be stressed that all incentives, including 
altruism, could only "open the door" to competitive employment; the 
participant" • performance once on the job almost certainly determined 
whether he or she kept the job. 
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CHAPTER V 

PARTICIPANT PERFORMANCE AMD PLACEMENT OUTCOMES 



This chapter examines two Matures of program success : employers' 
assessments of participants' work performance in Phase 2, and Phase 3 place- 
ment rates. Unquestionably, the judgments employers form about 
participants have a number of important implications. Employers are, of 
course, unlikely to give permanent jobs to participants whom they see as 
poor performers. At the same time, they will probably be unwilling to take 
on nev participants. Some may also convey their unfavorable opinions to 
other employers, who, in turn, may be reluctant to hire members of this 
population. Obviously, positive judgments can lead to very different and 
much more positive results. 

Placement rates indicate the extent to which sites were able to move 
participants into competitive jobs. While they do not present a complete 
picture of the program's employment effects (which will be better seen in 
the final impact analysis), they are one useful measure of program perform- 
ance. 

I. Employer Perspectives on Phase 2 Work Performance 

Employer perspectives were measured in a series of questions in the 
employer survey. The respondents were the direct supervisors of Phase 2 
participants. Before considering the findings from this study, one should 
first understand how the survey was structured. 

First , for most of the results, the unit of analysis was the 
participants evaluated, not the supervisors who evaluated them. (Some of 
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the supervisors had worked with, and therefore attested, more than one 
STETS participant.) Ce'cond, the evaluation covert only Phase 2 
participants, an; thus reflect t their performance as trainees, not as 
regular workers. Third, supervisors" retpontet were included only if they 
had worked with participant for at leatt one reonth, to ensure that they 
had a basis for forming judgments. fin fact, nlmost three-fourths of the 
participant?, evaluated had been assigned to the same supervisor for seven 
weeks or longer.) Finally, in certain cases, supervisors were ashed to 
compare participants to other workers who had held similar jobs for about 
the same amount of time. 1 (This was to ensure that STETS trainees were not 
unfairly compared to workers who had more experience or very different 
responsibilities. ) 

The results of the survey paint a fairly positive portrait. Panel A 
in Table 5.1 shows, for example, that a substantial majority of partici- 
pants were rated "as good as" or "better than" other workers on each of 
five important dimensions of performance: attendance, punctuality, motiva- 
tion, productivity and quality of work. Additionally, as shown in Table 
5.2, employers judged that 91 percent of participants got along "very well" 
or "fairly well" with co-workers. Finally, employers and supervisors were 
asked whether they would again sponsor, or favor sponsoring, a STETS parti- 
cipant knowing what they do now about participant performance. Eighty-six 
percent said that they would. 

It is important to note that the sample might have been biased in 
favor of the better workers. Since the survey included only participants 
who had worked for the same supervisor for at least a month, those excluded 
might have received more negative evaluations. Some possibly were removed 
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TABLE 5.1 

A. PERCENT OF PHASE 2 PARTICIPANTS WHOSE WORK PERFORMANCE WAS RATED 
G00D AS" OR "BETTER THAN" OTHER NEW EMPLOYEES IN SIMILAR 
JOBS, BY DIMENSION OF PERFORMANCE 



Dimension of Performance 


Percent of Participants 


Mteuaince 


85 .0 


vetting co work U»i Tine 


89.0 


Motivation 


79.0 


Amours of Work Done (Productivity) 


69.0 


Quality of Work Done 


78.0 


Number of Participants 


100 


• PERCENTAGE DISTRIBUTION OF NUMBER OF PERFORMANCE DIMENSIONS ON 
WHICH PARTICIPANT WAS RATED "AS GOOD AS" OR "BETTER THAN" 
OTHER NEW EMPLOYEES IN SIMILAR JOBS 


Number of Dimensions* 


Percent of Participants 


None 


1.0 


One 


3.0 


Two 


13.1 


Three 


11.1 


Four 


17.2 


Five 


54.2 


Total 


100 .0 b 


Number of Participants 


99 C 


(continued) 
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TABLE 5.1 (continued) 



SOURCE: MDRC tabulations of data from supervisor responses tc the 
STETS Employer Survey Instrument. 

NOTES: The survey was administered to employers of Phase 2 
participants at all five STETS sites between December 1982, and April 1583. 

*The performance dimensions include Attendance, Getting to 
Work On Time, Motivation, Amount of Work Done (Productivity), and Quality 
of Work Done. 

b 

Percentage distribution may not add exactly to 100.0 because 

of rounding. 

c 

Excludes one participant for whom no information was 
available on any performance dimension. 



no 



TABLE 5.2 



PERCENTAGE DISTRIBUTION OF SUPERVISOR ASSESSMENTS 
OF PHASE 2 PARTICIPANTS' RELATIONSHIPS WITH CO-WORKERS, AND TYPES CF PROBLEMS 
EXPERIENCED BY PARTICIPANTS WITH CO-WORKERS 



Item 


Percent of Participants 


Hov Well Participant Got Alone With Co-Workers 
Very Well 
Fairly Well 
Not Too Well 
Poorly 
Total 


57 .0 

O / A 

34.0 
7.0 
2.0 
100.0* 


Number of Participants 


100 


Participant Had Problem With Co-Workers 
Yea 
No 

Total 


35.0 
65.0 

1 AA A^ 

100 .0 


Number of Participants 


1 00 


Problems Experienced By Participant^ 
'Participant upset by teaaing by co-workers 
Participant had difficulty communicating with co-workers 
Participant got into fights or arguments with co-workers 
Participant was shy, quiet, or afraid to associate with 
co-workers 

Co-workers upset by participant's poor work or performance 
Co-workers felt that participant asked too many questions 
Co-workers offended by participant's poor hygeine 


22.8 
14.9 
11.4 

11.4 
29.6 
5.7 
5.7 


Number of Participants 


35 



SOURCE: MDRC tabulations of data from supervisor responses to the STETS 
Employer Survey Instrument, 

NOTES: The survey was administered to employers of Phase 2 participants at 
all five STETS sites between December 1982 and April 1983. 

'Percentage distributions may not add exactly to 100.0 because of 

rounding. 

b fiased on participants who were assessed as having had a problem with 
co-workers. More than one problem was identified for some participants. 

v 
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from the program at an earlier point because they were performing poorly. 

The survey evaluation could also reflect response bias on the part of 
the supervisors: Supervisors could have been less critical of STETS 
participants out of sympathy for the handicapped. Generally, this did not 
seem to happen. Further analysis of the data suggests that supervisors 
attempted to make meaningful assessments of performance. For example, when 
all six performance measures were included in a correlation matrix, the 
degree of correlation between variables was modest, at best (Table A. 2). 
In other words, supervisors tended to give both positive and negative 
evaluations to a participant depending on the performance ar^a measured — 
a pattern that lends credibility to the responses. 

Other data support the reliability of the assessments . As indicated 
in Table 5.3, which is based on MIS data, over 66 percent of the Phase 3 
placements were rollovers from Phase 2, testimony to employers' satisfac- 
tion with the performance of many participants. In addition, the value of 
output study, which is part of the forthcoming benefit-cost analysis, 
shows that participants produced goods and services of significant value in 
both Phases 1 and 2. 

The employer survey showed, however , that in comparison to other new 
employees in similar jobs, many participants had important work-related 
problems. As already noted, a good number of participants were not given 
positive evaluations on all dimensions of work performance. This is appar- 
ent not only from the correlation matrix, but also from Panel B in Table 
5.1, which indicates that only 54 percent of the participants received 
positive ratings on all five performance measures. 

Productivity proved to be the area of greatest difficulty. As is 
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TABLE 5.3 



PERCENTAGE DISTRIBUTION OF CHARACTERISTICS OF PHASE 3 PARTICIPANTS- 
FIRST JOB IN PHASE 3, BY PROGRAM SITE 



Characteristic 


Los 

Angeles 


Hew 
Tork 


Cin- 
cinnati 


St. 
Paul 


Tucson 


All 
Sites 


Hours Per Week (X) 
Fever Than 10 Hours 
10 - 20 Hours 
21 - 30 Hours 
Over 30 Hours 


0.0 
26.3 
10. 5 
63.2 


10.3 
27.6 
6 .9 
55.2 


12.5 
25.0 

50 .0 


15.4 
23.1 
JU .8 
30.8 


0.0 
21.4 
32 .1 

46,4 


7.1 
24.8 
17 .7 
50 .4 


Average Hours Per Week 


32.6 


27.5 


27 .1 


24.1 


30.5 


28.5 


Starting Wage Per Hour (X) 
Less Than $3.35 
$3.35 - $3.50 
$3.51 - $4.00 
Over $4.00 


0.0 
36.8 
26.3 
36.8 


3.4 
48.3 
24.1 
24.1 


0.0 
91 .7 
8.3 
0.0 


0.0 
46.2 
30.8 
23.1 


10.7 
78.6 
10.7 
0.0 


3.5 
62.8 
18.6 
15.0 


Average Starting Wag- Per 
Hour ($) 


4.23 


3.70 


3.38 


3.70 


3.33 


3.63 


Medical Benefits (X) 
No 

Yes, Immediate 
Yes, Delayed 


36.8 
47 .4 
15.8 


41.4 
34.5 
24.1 


16.7 
12.5 
70.8 


46.2 
30.8 
23.1 


71.4 
7.1 
21.4 


43.4 
24.8 
31 .9 


Sector (X) 
For-Prof it 
Non-Prof it 
Public 


52.6 
21 .1 
26 .3 


M R 

OA .0 

6.9 
10 3 


0/ »j 

12.5 
n n 


84 .6 
15.4 
U .0 


89.3 
7.1 
3 .6 


80 .5 
11.5 
8.0 


Occupational Category (X)* 
Clerical, Sales 
Service 
Agriculture 
Processing 
Machine Trades 
Benchwork 
Structural Work 
Miscellaneous 


21.1 
57.9 
0.0 
0.0 
0.0 
15.8 
0.0 
5.3 


55.2 
34.5 
0.0 
0.0 
0.0 
3.4 
6.9 
0.0 


0.0 
70.8 
0.0 
4.2 
4.2 
16.7 
0.0 
4.2 


7.7 
69.2 
0.0 
0.0 
7.7 
15.4 
0.0 
0.0 


0.0 
46 .4 

7.1 

7.1 
10.7 
10.7 

0.0 
17.9 


18.6 
53 1 
1.8 
2.7 
4.4 
11.5 
1.8 
6.2 


Union Job (Z) 


15.8 


34.5 


0.0 


0.0 


0.0 


11.5 


Rollover from Phase 2 (Z) 


36.8 


41 .4 


87.5 


100 .0 


78.6 


66.4 


Number of Participants 


19 


29 


24 


13 


28 


113 b 



(continued) 



TABLE 5.3 (continued) 



SOURCE: MDRC tabulations of Monthly Ststus Change Forms in the STETS 
iDrornation System. 

rounding Percent ** e di'tribution. may not add exactly to 100.0 because of 

m r • . . *°««P»tional categories were derived from groupings of similar jobs 

pubH ne" by y th e ^s' lltllttlTt TJ^Vf 9f liUii, 4th Ed^ion, 

puoii.ned by the U.S. Department of Labor, Employment and Training Administration, in 

data are ^2^^" P " tici P" t! in *>rk and one in Tucson for whom 
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apparent from Table 5.1 (Panel A), supervisors reported that 31 percent of 
the participants had more productivity problems than other new workers in 
similar jobs. Quality of work was another problem area: 22 percent of the 
participants were rated lover than other new workers. Among participants 
with productivity problems, slow performance was the reason in over half of 
the cases (Table A. 3). Other difficulties mentioned were understanding, 
remembering or doing tasks (in about 38 percent of the cases) and poor moti- 
vation or concentration (in almost one-quarter of the cases). Supervisors 
cited poor work quality as stemming from motivational and attitudinal prob- 
lems in 19 percent of the cases, and from difficulty in understanding or 
remembering the tasks in one-third. 

As Table 5.2 indicates, 35 percent of all participants in the analysis 
sample had some difficulties in co-worker relations. Such strains, of 
course, are not uncommon among non-handicapped employees. For the STETS 
participants who experienced such problems, two types vere most common: 
Participants became upset by co-workers" teasing (23 percent), and co-work- 
ers had trouble dealing with the poor performance or work of participants 
(30 percent) . 

Supervisors were also asked to estimate the amount of time it took 
them (and others in the organization) to train STETS participants compared 
to other new employees in similar jobs. They reported that 58 percent of 
the participants in the sample required "much more" or "somewhat more" 
training time (Table 5.4). One reason predominated: In about 66 percent of 
the cases, participants had difficulty understanding or remembering tasks. 

For the three measures of work performance on which participants 
showed considerable variation — productivity, quality and training time — 
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TABLE 5.4 



PERCENTAGE DISTRIBUTION OF TIME SPENT BY SUPERVISORS 
TO TRAIN PHASE 2 PARTICIPANTS COMPARED TO OTHER NEW WORKERS IN SIMILAR JOBS 
AND REASONS FOR MORE TRAINING TIME REQUIRED 



Item 


Percent of 
Participants 


Relative Amount of Training Time 
Much More Time 
Somewhat More Time 
About the Same Time 
Less Time 
Don't Know 
Total 


24.0 
34.0 
35.0 
6.0 
1.0 
100.0* 


numoer 01 rarcicipants 


100 


Reasons For ParticioAnts R^fluirinv vrnn Timp^ 

Participant had difficulty understanding/remembering tasks 
Supervisor needed more time to evaluate participant 
Participant lacked reading, writing, math skills 
Participant had physical problems 
Participant had lover motivation 
Participant was more likely to make mistakes 
Participant had poor concentration 
Participant asked more questions 
Other 


65.5 
31.0 
6.9 
1.7 
5.2 
5.2 
3.5 
6.9 
6.9 


Number of Participants 


58 



SOURCE: MDRC tabulations of data from supervisor responses to the STETS 
Employer Survey Instrument. 

NOTES: The survey was administered to employers of Phase 2 participants at 
all five STETS sites between December 1982 and April 1983. 

^Percentage distribution may not add exactly to 100.0 because of 

rounding. 

Based on participants who required "Much More" or "Somewhat More" 
training time. More than one reason was cited for some participants. 
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researchers conducted a further statistical analysis (cross-tabulation) to 






determine whether ratings were associated with certain background charac- 






teristics: age, sex, race, level of retardation, presence of a secondary 






handicap, possession of a high school diploma or equivalency degree, any 






prior work experience* and receipt of SSI or SSDI at enrollment. Most of 






these variables were not significantly associated with differences on the 






performance measures. For example, differences in IQ level in both the 






mild and borderline categories (into which about 90 percent of participants 




fell) did not seem to influence performance, a finding consistent with a 






good deal of previous research in the field. * (However, it is difficult to 






draw conclusions from these data about moderate retardation and the level 






of performance.) 






One background variable that produced statistically significant 






differences on all three performance measures was race.^ In each case, 






* 

whites performed less well than non-whites. One speculation is that non- 






whites were more frequently mislabeled as retarded, an observation which 






some staff members had made in their own informal assessments of 






participants' work performance.^ 






Two main conclusions can be drawn from these supervisor assessments. 






First, they suggest it is feasible to depend on outside employers to pro- 






vide training in real work settings. As a group, STETS participants were 






not an excessive burden to supervisors and they appeared to have performed 






well enough so that they showed promise for competitive employment. At the 






same time, it is clear that many participants had important difficulties in 






Phase 2, indicating the continued need for support at this stage. 






The employer survey also shows that, for the most part, Phase 2 super- 
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vifcors did not resent counselors' visits to the worksites. Eighty-seven 
percent reported that "no problems irope" from these visits. In most 
cases, too, the supervisors did not react negatively to the amount of time 
counselors spent with participants during office visits — which, for two- 
thirds of the participants, was less than one hour. (See Table A. 4.) Super- 
visors instead tended to view the counselors' assistance as helpful. For 
64 percent of the participants, supervisors rated the support as "very help- 
ful, M and for 14.4 percent, "somewhat helpful." 

II. Placement Into Unsubsidized Jobs 

This chapter has so far focused on the performance of participants as 
Phase 2 trainees. It now turns to a consideration of the extent to which 
the program was able to help these trainees become regular worker". 

The demonstration's MIS data on types of departures from the program 
(Table 5.5) show that about 42 percent of all enrollees were placed into 
unsubsidized Phase 3 jobs. An additional 9 percent were placed into other 
educational and training programs, subsidized employment or sheltered work- 
shops. 

It should be noted that the demonstration's method of calculating 
placement rates — focusing on all participants ewer enrolled — is a 
strict one, for it includes all individuals the program ever attempted to 
assist. Other employment programs have sometimes utilized less stringent 
measures. For example, in some programs, individuals who were enrolled but 
who withdrew from the program at the outset or who were terminated by the 
program prior to being assigned to a training job are not included in the 
population on which placement rates were based. This, of course, would 
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TABLE 5.5 

PERCENTAGE DISTRIBUTION OF TYPES OF DEPARTURES FROM STETS 
AMONG ALL ENROLLEES, BY PROGRAM SITE 



Type of Departure 


LOI 

Angeles 


new 
York 


Cin- 
cinnati 


St. 
Paul 


Tucson 


All 
Sites 


Entered Untubt idized 
Employment 


33.3 


49.2 


33.7 


28.9 


55.8 


41.6 


Entered Sheltered Workshop 


7.0 


1.5 


1.6 


8.9 


3.8 


4.3 


Entered Educational or 
Training Program or Other 
Subsidized Employment 


7.0 


1.5 


4.8 


2.2 


9.6 


5.0 


Terminated But Not Placed 
Into an Educational or 
Training Program or a Job 




47 .7 


54.8 


60.0 


30.8 


49.1 


Total 


100 .0 


100.0 


100.0 


100.0 


100.0 


100.0 


Number of Enrol lees 


57 


65 


62 


45 


52 


281* 



SOURCE: MDRC tabulation, of Monthly Statu. Change For*, in the STETS 
information System. 

NOTES: Percentage distributions may not add exactly to 100.0 because of 
rounding • 



in T.™ f J Xcl " d " one P«ticipMt in Los Angeles, one in St. Paul and one 
in Tucson for whom data are missing. 
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lead to higher placement rates. In STETS, for example, when only parti- 
cipants who reached Phase 2 are considered, the placement rate is 61.4 
percent. 

All sites were not equally successful in placing clients into unsub- 
sidized jobs, with placement rates varying widely, from a high of 56 
percent in Tucson to a low of 29 percent in St. Paul. Some of the reasons 
for this site variation will be explored later in this chapter. 

An additional analysis using MIS data examined vhether placement rates 
varied according to selected background characteristics, including age , 
sex, level of retardation, ethnicity, prior employment experience and prior 
job training. The results *re presented in Table 5.6 and shew some inter- 
esting differences. For example, placement rates were highest among 
persons classified as borderline retarded, and lowest — although still 
sizable — among the moderately retarded (a small subgroup containing only 
36 individuals). Other differences were found in the sex and ethnicity 
variables, with males and Hispanics showing higher placement rates than 
females and other ethnic groups, respectively. 

In general, however, the placement rates by subgroup were not 
dramatically different from the average for the demonstration as a whole. 
One can conclude that it is thus difficult to predict an enrollee's 
likelihood of finding a job on the basis of these objective characteristics 
alone — a conclusion that is consistent with the findings from the 
employer survey. Together, these two studies suggest that it is probably 
unwarranted to target future STETS programs to subgroups distinguished by 
the characteristics studied here. The issue of targeting will be further 
explored in the demonstration's impact analysis, which will examine the 
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TABLE 5.6 



PLACEMENT RATE INTO AN UNSUBSIDIZED JOB, 
BY SELECTED CHARACTERISTICS OF ENROL LEES 
AT TIME OF ENROLLMENT 



Characteristic 


Placement Rate 


Age 

18-21 Years 
22-24 Year. 


41 .4 
42 .4 


rc tirai cion Range 
Moderate 
Mild 

Borderline 


33.3 
38.6 
49.1 


Any Secondary Handicaps 
Yes 
No 


39.3 
43 .4 


Sex 
Male 
Female 


49.1 
30.7 


Ethnicity 
wnite 
Black 
Hispanic 
Other 


40.4 
39.5 

50.0 


Any Prior Job Training 
Yes 
No 


42.6 
41 .3 


Any Prior Employment 
Yes 
No 


41 .3 

42 .7 


Number of Enrol le<s 


283* 



SOURCE: MDRC tabulations of 
Application/Enrollment Forms in the STETS 
Information System. 

NOTE: *Excludes one enrollee for 
whom data are missing. 
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influence of various background characteristics on post-program employment 
and other behavioral outcomes. When sample sites permit , that analysis 
will compare outcomes for various matched subgroups of individuals in the 
experimental and control groups. 

While placement rates are useful in assessing how veil STETS vas able 
to achieve its short-term goal of moving clients into competitive employ- 
ment! by themselves they are only a limited measure of program performance* 
In order to understand whether the 42 percent demonstration-vide placement 
rate is a "good" outcome, it is important to ask tvo additional questions: 
1) Hov does this rate compare vith placement rates achieved by other 
programs; and 2) Hov do the post-program employment outcomes of STETS 
participants compare vith the outcomes these individuals vould have 
achieved vithout the assistance of the program? Ultimately, it is the 
snsver to the second question, to be addressed by the impact analysis, that 
matters most: No matter hov high the placement rate, the program vill not 
show a net impact if members of the control group do as veil as or better 
than members of the treatment group. Conversely, even if less than a 
majority of participants are successfully placed into unsubsidited jobs, as 
vas true for STETS, the program may prove to have a positive net impact if 
members of the control group are less successful. 

In the absence of post-program impact data,^ only the first question 
posed above vill be addressed here: Hov does the STETS placement rate 
compare vith that of other programs? Ideally, other employment programs 
for mentally retarded persons vith characteristics similar to those of 
STETS participants should be used for comparative purposes. Unfortunately, 
there is only a small quantity of such published program data and, among 
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these sources, considerable variation on how rates were calculated* None- 
theless, it is possible to make some comparisons. 

Studies of sheltered workshops show that these organizations tend to 
place few clients into competitive employment. As noted in Chapter I, a 
Department of Labor study found that, in 1976, workshops placed into 
competitive jobs approximately 8 percent of the average number of clients 
who were in daily attendance, including non-retarded as well as retarded 
clients (U.S. Department of Labor, 1979). Somewhat higher rates were found 
in the state-federal Vocational Rehabilitation program. In a special study 
prepared for MDRC by Mathematics Policy Research, it was determined that, 
of the approximately 140,000 retarded persons served by this program in 
fiscal 1978, 11 percent (or 28 percent of the cases of retarded persons 
officially "closed" that year) were placed into competitive jobs (Burg- 
hmrdt al,, 1980). 

An alternative comparison can be made with the outcomes achieved by 
the National Supported Work Demonstration, which served as the genesis of 
STETS. The target groups in that demonstration, which were followed from 
1975 through 1978, Included long-term female welfare mothers, ex-addicts, 
ex-offenders and disadvantaged youths. While not mentally retarded, the 
members of each group had poor work histories and were considered very 
difficult to employ. While comparisons must be cautious — given the 
differences in target groups, dates of operation and program features — it 
is noteworthy that the STETS placement rate exceeded Supported Work's over- 
all 29 percent placement rate for all target groups. The STETS rate even 
surpassed that for welfare mothers, the target group that showed the 
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most poiitive poit-program employment and benefit-coat outcomes (MDRC 9 
1980). 

III. Characteristic! of Phage 3 Jobs 

Information on the characteriitici of the Phase 3 jobs obtained by 
STETS sites Has been presented in Table 5.3. 8 As that table shows, the 
vast majority (81 percent) of participants who were placed into unsubsi- 
dized jobs entered for-profit firms. Over half obtained service jobs, such 
as janitorial and kitchen positions, while 19 percent entered clerical wosl 
and 12 percent found benchvork (e.g., assembly) jobs. Few placements (12 
percent) were union jobs, and most of these were made by the New York site. 

In several sites, job developers, after some experience, stopped 
looking for, or reduced their emphasis on, jobs in fast-food restaurants. 
With their high turnover rates, these jobs at first seemed promising, but 
the job developers concluded that participants usually could not work with 
t'-r speed these jobs demanded. Only those assigned to low-pressure tasks 
(such as keeping the salad bar stocked) managed to perform satisfactorily. 

On average, Phase 3 jobs provided 28.5 hours of work per week, meaning 
that there were fewer full-time jobs than had been anticipated. This was 
because full-time work was difficult to find, although in some instances 
participants preferred part-time jobs or staff thought it more appropriate 
for them. 

Phase 3 wages generally paid above the federal minimum. Only 4 per- 
cent of participants earned lest than $3.35 per hour. The average starting 
wage was $3.63 per hour, and 34 percent of the participants were paid over 
$3.50 per hour. Over half of the participants (57 percent) had joba provid- 
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ing medical benefits, either immediately or after a specified period of 
time. These benefits were particularly important, given the participants' 
secondary disabilities.^ 

Available MIS data show that, among participants contacted during the 
six-month period in Phase 3, the majority were still employed, usually at 
the same firm or agency. 10 Many had worked for the same employer in Phase 
2; as shown in Table 5.3, over 66 percent of the placements were rollovers. 
These data suggest that a considerable number of participants were able io 
hold unsubsidized employment for at least several months. Additional data 
on job retention will be presented in the impact analysis. 

IV. Non-Positive Terminations from STETS 

While the placement data are encouraging, over half of the partici- 
pants were not placed into unsubsidized jobs, nor were they linked 
(directly by STETS) with other educational or training programs (Table 
5.5). The analysis considers such terminations to be non-positive; that 
is, although they were not necessarily negative for the participants, they 
did not result in competitive employment, the program goal. The reasons 
given for the terminations suggest that a good many participants had 
employment problems that could not be overcome, even with program 
assistance. 

Table 5.7 separates out the reasons for termination. It indicates, 
for example, that only 12 percent of the terminations were caused by the 
inability of participants to perform assigned tasks adequately. This is 
consistent with a common finding from the mental retardation literature, 
which suggests that only a minority of mildly retarded persons lose jobs 
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TABLE 5.7 



PERCENTAGE DISTRIBUTION OF REASONS FOR NON-POSITIVE TERMINATION 
FROM STETS, BY PROGRAM SITE 



Reason 


Los 

Angeles 


Nev 
York 


Cin- 
cinnati 


St. 
Paul 


Tucson 


All 
Sites 


Ilnahl* To Perform Job Tasks 


0.0 


16.1 


8.8 


25.9 


12.5 


12.3 


Disruptive Behavior, 
Unsatisfactory Attendance. 
Personal Problems 


13.3 


38.7 


35.3 


33.3 


56.3 


33 .3 


Vmt t \ r \ nint DiiiAtiif With 

i li L.Xl.4>fS*llL. i/AVWk&V&AGU Willi 

Program or Intends To Enter 

Oth#T* TAwcm. t i on a 1 or 

Training Program 


10.0 


3.2 


2.9 


14.8 


12.5 


8.0 


Personal Health Problems 


13.3 


12.9 


2.9 


7.4 


6.3 


8.7 


Moved 


6.7 


0.0 


5.9 


3.7 


0.0 


3.6 


Loss of SSI or Other 
Transfer Benefits 


0.0 


0.0 


5.9 


0.0 


6.3 


2.2 


Program Lost Contact 
With Participant 


10.0 


0.0 


8.8 


0.0 


0.0 


4.3 


Death 


0.0 


0.0 


0.0 


3.7 


6.3 


1.4 


Other 


46.7 


29.0 


29.4 


11.1 


0.0 


26.1 


Total 


100.0 


100 .0 


100 .0 


100.0 


100.0 


100.0 


Number of Enrol lees 


30 


31 


34 


27 


16 


138* 



SOURCE: MDRC tabulations of Monthly Status Change Form* in the STETS 
Information System. 

MOTES: Percentage distributions may not add exactly to 100.0 because of 
rounding. 

'includes all enrol lees vho vere terminated from STETS but not placed 

into an unsubsidized job or into an educational or training program. 
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became they cannot adequately perform the tasks involved. Many more 

terminations (33 percent) were caused by disruptive behavior on the job, 

unsatisfactory attendance, or personal and family problems. Moreover, 8 

percent of the non-positive terminations occurred because participants vere 

dissatisfied with the program or preferred another type of education or 

training program (although, in these instances, they vere not placed in 

those programs by STETS staff). In a few cases, participants vere 

terminated (by themselves or their families) because they believed they 

might lose SSI or other income transfer benefits. 

Distinctions were not always clear-cut, and many participants left for 

a combination of reasons. The following description of circumstances 

leading to the termination of one participant illustrates thifc point: 

Paul could do a good job for short periods of time but was 
unable to sustain this over a full day. He was hyperactive 
and "acted up." He would sometimes yell and pick up chairs. 
He also started having an attendance problem: Either he 
wouldn't show up or he would leave in the middle of the day. 
The STETS staff met with his Vocational Rehabilitation 
counselor and his mother a number of times about this problem 
but it didn't seem to help. His mother was concerned, but had 
no control over him. 

Usually non-positive terminations happened fairly soon after program enroll- 
ment; 72 percent took place before Phase 2 started. 

V. Factors Affecting Placement Outc ome 

Tliis section will identify conditions that may have affected place- 
ments and contributed to site differences in those rates. first, however, 
it should be noted that site performance was affected simultaneously by 
many factors which are difficult to disentangle -- among them, the 
capabilities of enrollees, training strategies, job development techniques 
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and local economic conditions* Nevertheless, it is possible to identify 
some that probably played a strong role in the outcomes. 

One key constraint was that, given the program's broad eligibility 
criteria, many participants were not sufficiently capable, prepared or 
motivated to benefit from it. A second constraint, previously discussed, 
was the truncation of the program. While the implementation experience was 
generally positive, it is reasonable to speculate that more participants 
would have been placed had the program been given time to mature. For 
example, in the National Supported Work Demonstration, the overall place- 
ment rated improved from 24 percent in the first year to 36 percent in the 
last year. While this may reflect improved economic conditions towards the 
end of that period, greater staff experience was probably also a factor. 

As indicated by the difficulties in St. Paul and Los Angeles, weak job 
development strategies contributed to lower placement rates at some sites. 
The poor economic environment was another limiting factor although, as 
shown by the St. Paul site (where a relatively low unemployment and place- 
ment rate both prevailed), a stronger economy did not necessarily guarantee 
better results. 

A number of other factors initially thought to be important turned out 
not to be very decisive. For example, the use of a Phase J sheltered or 
non-sheltered strategy could have influenced placement rates. As Chapter 
III has shown, each had advantages and disadvantages, but both appeared to 
be feasible. The highest placement rates in the demonstration were 
obtained by Tucson — which used a sheltered strategy — and by New York, 
which took a non-sheltered approach. 

The training practices used by a program could also have affected 
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placement outcomes: e.g., the demonstration"! 500-hour limit on Phase 1 
paid activity. Early in the demonstration, some staff members felt that 
many participants needed more time in that phase. Their dissatisfaction 
raised the question of whether placement rates would have risen noticeably 



large, the available evidence does not support such a conclusion. While a 
short extension of Phase 1 might have changed outcomes for some poorly 
performing participants, staff believed that most participants probably 
would not have shown much improvement. As the director of the Cincinnati 
site remarked, "They'd either be ready in 500 hours ov 1,500 hours. 
Fiddling with a few hundred hours wouldn't make much difference. " 

Such observations suggest that participants who cannot be ready for 
Phase 2 in 500 hours may not be appropriate for the quick employment 
transition specified in the STETS model. A longer Phase 1 would also 
increase the costs of the program, since Phase 2 has a lower overall level 
of program services and more private sector coverage of participant wages. 
Moreover, a longer Phase 1 would have worked against the goal of moving 
participants expeditiously into higher-stress work positions. 

During the planning stages of the demonstration, MDRC was particularly 
interested in the capacity of the sheltered workshop, affirmative industry 
and state agency to operate the program adequately. It wan thought, for 
example, that the limited experience of these organizations with job devel- 
opment and outside placement might impede smooth implementation. This, 
combined with the fact that these organizations had other service commit- 
ments (such as longer-term sheltered employment or casework and referral 
services), raised the question of whether STETS might have to compete for 



if participants had spent more time in initial training activities. By and 
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top management attention and access to other organizational resources. 

To some extent , problems did arise , particularly in St. Paul. In 
fact, by the end of the demonstration! that site's director had become 
fairly pessimistic about the feasibility of jointly operating a sheltered 
employment and a transitional program "under the same roof. 11 The experi- 
ence of the Cincinnati site, however, suggests otherwise. The high-quality 
operation of that program implies that two missions can co-exist if each 
mission his a strong commitment from the parent organization. 

The experience in the Tucson site also shows that multiple organiza- 
tional goals need not undermine the implementation of a program like STETS. 
Moreover, Tucson clearly demonstrates that an employment program can be 
directly operated by a state agency, even when that agency's previous 
employment approach has generally been to refer mentally retarded persons 
to other agencies serving this group. 
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CHAPTER VI 

PRELIMINARY FINDINGS FROM THE IMPACT ANALYSIS 



I. Introduction 

While the analysis in this report hat shown that STETS is a feasible 
program to implement , it cannot be determined whether the program's 
participants have found more jobs or are better off in terms of other 
economic and social circumstances than they would have been had they not 
entered the program. To make that determination, an impact analysis will 
compare the behavior and experiences of the program's enrollees (the experi- 
mental group) to those of a control group, using a longitudinal survey in 
which both experimentals and controls are interviewed shortly after random 
assignment, and then at 6 , 15 and 22 months after that baseline interview. 
The overall sample sire for the study numbers almost 500 persons, although 
the six-month survey was administered to a randomly selected subsample com- 
prised of two-thirds of that group. * 

Because the impact study is still under way, a preliminary analysis 
has been conducted based on the six-month interviews. This analysis 
describes the job training, employment and other activities of the two 
groups at a time when most experimentals were still enrolled in the STETS 
program, thus providing some initial insight into the patterns of service 
receipt and labor market behavior of sample members at an early stage in 
the demonstration. The outcomes of this effort, and a general overview of 
the impact research design, are the subjects of this chapter. 

It is important to emphasize the preliminary nature of this analysis, 
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which hat two uain limitations even for the estimation of in-program 

impacts. First, che results are not adjusted statistically to control for 

any baseline differences between the experimental and control groups. 

Second, the overall site of this sample — 267 respondents (131 controls 

and 136 experimental) — is smaller than the six-month sample that will be 

used in the final analysis. Consequently, the findings presented here may 

2 

differ from those in the final impact report. 

II, Research Design and Data Collection Methodology for the Impact 



STETS impacts will be examined in four main areas: 1) labor market 
performance (e.g., employment, earnings and job-search activities); 2) par- 
ticipation in education and training programs; 3) use of transfer pro- 
grams (e.g., participation in SSI and SSDI and the amount of benefits); and 
4) social and living skills (e.g., living arrangements, family formation, 
money-handling skills and travel skills). (For a more detailed description 
of these variables, see Appendix C.) In this six-month analysis, only a 
few specific outcome measures have been selected for study: employment 
status; participation in training or school programs; financial management 
skills; and receipt of SSI or SSDI. 

The data collection strategy posed an initial problem in the impact 
study. Since a broad range of information on individuals' activities, 
personal characteristics , living arrangements and social supports is 
required to address the issues above, it was considered desirable to 
collect data directly through in-person interviews with sample members . 
However, because a large sample of retarded persons had never before been 
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as extensively interviewed, there were tone initial reservation! about the 
quality and completeness of self-reported data from this population. Not 
only do the mentally retarded have lower cognitive and verbal 
communications skills, some individuals have problems comprehending time 
and money concepts, which are critical in measuring labor market and other 
variables. Further, given the young age of the STETS group, many have only 
limited experience in communicating with strangers. Finally, the generally 
higher incidence of physical and/or emotional problems associated with 
mental retardation could conceivably create problems in obtaining reliable 
interview data. 

Thus, several possible data sources were considered and tested before 
the impact research began. In a pilot study conducted with early enrollees 
in three sites, data from three sources were examined: records kept by the 
local STETS agency, in-person interviews with the respondent (the mentally 
retarded sample member), and in-person interviews covering the same topics 
with a proxy respondent (parent, guardian or counselor).^ 

The pilot study found that a data collection strategy based on self- 
reported interviews with the STETS sample members was in fact feasible. 
Completion rates were high for both primary and proxy respondents (at 95 
and 99 percent, respectively), but the incidence of missing or incomplete 
data in the primary interviews was unexpectedly low. In light of these 
findings, in-person interviews with both the experimental and control 
groups were selected as the primary source of data, supplemented by proxy 
interviews when sample members could not provide complete or accurate 
information. (See Appendix C.) 
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III. The Six-Month Research Staple 

A. Characteristic! 

The six-month research sample was randomly selected from the full 
analysis sample, and it appeared to be representative . of that larger group. 
This conclusion is based on a statistical comparison (not included here) of 
each sample's distribution of baseline characteristics* 

Within the six-month sample, 18 percent of respondents vere located in 
Cincinnati, 21 percent in Los Angeles, 26 percent in Nev York, 12 percent 
in St. Paul, and 24 percent in Tuscon. (See Table 6.1.) IQ scores fell 
within the moderate range of retardation for 15 percent of the sample 
members, vithin the mild range for 48 percent, and within the borderline 
range for 36 percent. Males constituted 56 percent of the sample, and the 
average age at enrollment was 20 years. 

Consistent with the rules established for program enrollment, few 
members in the six-month sample held regular jobs at the time of program 
application. A large proportion (37 percent) received either Supplementary 
Security Income (SSI) or Social Security Disability Insurance (SSDl), by 
far the two most important sources of transfer income. Furthermore, a 
majority (70 percent) appeared to be lacking financial management skills. 

B. Activities of the Six-Month Sample 

This section describes and compares the experiences of experimental 
and control group members on the outcome measures mentioned earlier. The 
data are analyzed for each group separately and then compared. Data for 
subgroups — defined by demonstration site, IQ range, sex and age at 
baseline — are analyzed in the final section. 
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TABLE 6.1 



PERCENTAGE DISTRIBUTION OF CHARACTERISTICS OF THE SIX-MONTH 
ANALYSIS SAMPLE AT BASELINE, BY RESEARCH CROUP 



Characteristic 


Control 
Croup 


Ex; zrinental 
Croup 


Total 


Site 

Lot Angeles 
Nev York 
Cincinnati 
St* Paul 
Tucson 


19.8 
26 .0 

13.0 
22.9 


21.3 
25.7 
17 .6 
11.0 
24.3 




20.6 
25.8 
18.0 
12.0 
23.6 


Retardation Hint ^ 
Borderline 
Mild 

Moderate 


37 .4 
46.6 
16.0 


35.3 
50 .0 
14.7 


■ 


36.3 
48.3 
15.4 


Sex 
Male 
Female 


58.0 
42 .0 


53.7 
46 .3 




55.8 
44.2 


Age 

Under 22 Years 
22 Years and Over 


71.0 
29.0 


58.1 
41 .9 




64.4 
35.6 


Baseline Financial 
Management SVilla 

Not Independent 

Independent 


70.2 
29.8 


70.6 
29.4 




70.4 
29.6 


Baseline Receipt of 
SSI or SSDI 

Did Not Receive 

Did Receive 


61.7 
38.3 


64.3 
35.7 




63.1 
36.9 


Sample Sire* 


131 


136 


267 



SOURCE: Baseline and six-month surveys of the STETS impact analysis, 
administered by Mathematica Policy Research from December 1981 through 
February 1983. and April 1982 through June 1983. respectively. 

NOTES: The six-month sample is comprised of two-thirds of the full 
research sample for the impact analysis. 

Percentage distributions may not add up to 100.0 due to 

rounding. 

^Sample sizes for receipt of SSI/SSDI are 115. 126, and 241, 

respectively. 
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1 . Activities of the Control Group Sample 

Labor Market Activities , The control group sample vat 
fairly active at the fix-month point: At shown in the first column of Table 
0.2, over two-thirds reported some form of activity related to a job, train- 
ing or to school. Nearly half (48 percent) reported working in a job for 
vhich they were paid. (About 21 percent held a paid job as part of a train- 
ing or school program, 14 percent had a paid job in a vorkshc jt an activi- 
ty center, and 13 percent reported that they were employed in a regular 
job.) While paid jobholders in the control group worked an average of 27 
hours a week (Table 6.3), over one-third (37 percent) worked full-time, 
which is defined as over 32 hours per week. 

The average earnings per week (of those employed) was $56, for an 
average hourly wage rate of $2.07. While average hours worked was similar 
in all three categories of paid jobs, eai;;i..gs and estimated wage rates 
followed an expected pattern. Regular (or non-training) jobs paid the high- 
est wages; workshop and activity center jobs paid the lowest. 

Slightly over 8 percent of the control group sample reported that they 
were working but not receiving pay; 2 percent said they were not working, 
but were taking part in a training program; and almost 10 percent reported 
that they were not working but attending school (Table 6.2). A fairly 
large number o£ controls were inactive six months after random assignment: 
31 percent reported no job, school or training activities at that time. 

Independence Measures , Variables measuring the independence 
of sample members have also been examined. Financial management skills, as 
reflected in responses to several different interview questions, were 
exhibited by 28 percent of the control sample members at the time of the 
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TABLE 6.2 



SAMPLE MEMBERS' ACTIVITIES AT SIX MONTHS AFTER BASELINE, 

BY RESEARCH CROUP 



Activity 


Control 
Group 


Experimental 
Group 


Difference 


Percent 
Change 


Any raid Job (Z) 


48.1 


i£.4 


20.3** 


42.2 


tegular Job 
Training or School 

Job 
Workshop or 

Activity Center 


13.0 
21.4 
13.7 


11.0 
50.7 
6.6 


-2.0 
29.3** 
-7 .1* 


-15.4 
136.9 
-51.8 


Unpaid Job (Z) 


8.4 


2.2 


-6.2** 


-73.8 


Training with No Job (I) 


2.3 


5.9 


3.6 


146.5 


School with No Job (Z) 


9.9 


2.9 


-7.0** 


-70.7 


inactive: No Job, 

Training, or School(Z) 


31.3 


20.6 


-10.7** 


-34.2 


Any Job Training (X) 


40.0 


61.8 


21.8** 


54.5 


Any School (Z) 


19.8 


7.4 


-12.4** 


-62.6 


Financial Management 
Skills (Z) 




3/ .5 


9.3 


33.0 


Receipt of SSI or 
SSDI (Z) 


30.2 


27.9 


-2.3 


-7.6 


Average Monthly Amount 
of SSI or SSDI for 
Those Who Received 
Benefits ($) 


251 


260 




3.6 
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SOURCE: Six-month follow-up survey of the STETS impact analysis 
administered by Mathematic. Policy Research from April 1982 through June 

.«,, I?? 1 ?' u "" Ple f ° r th " • n « 1 y» i » " 131 for the control group 

and 136 for the experimental group. 

"Regular jobs include all paid jobs of over 4 hours per week 
which are not specified as training or school jobs or in workshops or 
activity centers. See Appendix C for a complete description of how the 
outcome measures were defined. 

♦Statistically significant at the 10 percent level. 
♦♦Statistically significant at the 5 percent level. 
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TABLE 6.3 



CHARACTERISTICS OF SAMPLE MEMBERS' JOBS AT SIX MONTHS AFTER BASELINE, 
BY TYPE OF JOB AND RESEARCH GROUP 



Tvn» ftf TaK mr\ A 

A /F™ v£ JOD InQ 

Characteristic 


Control 
Group 


Experimental 
Group 


Difference 


Percent 
Change 


Any Paid Job 

Average Hours Per Week 
Average Earnings Per 
Week ($) 

Average Wage Per Hour ($)* 


26.8 

c c c c 

55 .55 

2.07 


29.6 

76.54 
2.59 


2.8* 

20.99** 
0.52 


10.4 

37.8 
25.1 


Sample Size 


63 


93 






Regular Job a 
Hours 

E M rn inffc ( ^ ^ 

Wage Rate ($) 


27.3 

DO 1 O 

88.13 
3.23 


34.6 
108.26 
3.13 


7.3** 
20.13 
-0.10 


26.7 
22.8 
-0.0 


Sample Size 


17 


15 






Training Job 
Hours 

P a v*i i « r> e ( ^ 

Lirnin^s \y/ 

Wage Rate ($) 


25.3 
55.26 
2.18 


28.8 
73.86 
2.56 


3.5 

18.60** 
0.38 


13.8 
33.7 
17.4 


Sample Size 


28 


69 






Workshop or Activity Center 
Hours 

Earnings (.$) 
Wage Rate ($) 


28.7 
25.24 
0.88 


26.9 
44.21 
1.64 


-1.8 
18.97** 
0.76 


-0.1 
75.2 
86.4 


Sample Size 


18 


9 







pay, 



SOURCE: Six-Month follow-up survey of the STETS impact analysis, 
administered by Mathematica Policy Research from April 1982 through January 
1983, J 

NOTES: Table includes only sample members who reported working for 

'Average wage per hour is calculated directly from average 
earnings and hours per week; no significance tests were computed. 

b Regular jobs include all paid jobs of over 4 hours per week 
which are not specified as training or school jobs or in workshops or activity 
centers. See Appendix C for a complete description of how the outcome measures 
were defined. 



♦Statistically significant at the 10 percent level, 
♦♦Statistically significant at the 5 percent level. 
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six-month interview. However, 30 percent of the sample received assistance 
from SSI or SSDI. The average monthly SSI and SSDI benefits paid to 
control group members was $251* 

2 . Activities of the Experimental Group at Si x Months 

Labor Market Activities , As shevr: in the second! column of 
Table 6.2, almost 80 percent of the experimental group members were active 
at the six-month point. Over 68 percent of the sample had a paying job. 
Over half of the sample held paid jobs in training programs, 11 percent had 
regular (non-training) jobs, and only 7 percent reported positions in 
workshops or activity centers* 

As shown in the second column of Table 6.3, jobholders in the 
experimental group worked an average of almost 30 hours per week (one-half 
worked full-time), with average weekly earnings of $77. The hourly wage 
rate was $2.59. 

From Table 6.2, it can be seen that very few of the experimental group 
(2 percent) reported that they were working for no pay. Almost 6 percent 
said they were not working, but were participating in training programs, 
and 3 percent reported that they were not working but attending school. 
Thus, only one-fifth of the treatment group sample was inactive six months 
after random assignment. 

Independence Measures . Evidence of financial management 
skills was exhibited by over one-third (38 percent) of the experimental 
group. Receipt of benefits from either the SSI or SSDI program was report- 
ed by 28 percent of the members. The average monthly amount of combined 
benefits for those receiving the payments was $260. 
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3. 



Comparison of Experimental and Control Group Activities 



Labor Market Activities. As seen in Table 6.2, 31 percent 



of the control group compared to almost 21 percent of the experimental 
were inactive (i.e., not engaged in a job, training or school program) , 
representing a reduction of 34 percent. Put another way, the employment 
and training activity level of the experimental group was over 10 percent- 
age points higher than that of the control group. 

An even greater distinction can be seen in the distribution of 
activities between the two groups* The most substantial difference was in 
the proportion holding a paid job as part of a training program: The 
experimental rate was higher than that of the control group by 29 
percentage points (an increase of 137 percent), undoubtedly reflecting this 
group's enrollment in STETS at the six-month point. This group's workshop 
participation was also influenced by STETS because of the sheltered Phase 1 
training approach used in St. Paul, Cincinnati and Tucson. The 
experimental rate of workshop activity was, however, considerably lower 
than that of the control group. 

Offsetting these training-related differences are notably lower 
experimental group participation rates in unpaid jobs and in school, and a 
slightly lower rate of employment in regular jobs; the latter activity was 
largely precluded by the continued participation of STETS participants in 
the program at six months after enrollment. However, when all paid employ- 
ment is considered, experimental group members worked 10 percent more hours 
and earned 38 percent more dollars a week than the control group (Table 
6.3). Moreover, full-time work (i.e., over 32 hours per week) increased by 
over 38 percent.^ 
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demonstrated by more members of the experimental group than the control 
group (a difference of 9 percentage points, or an increase of 33 percent). 
This difference may reflect in part the greater opportunity of this group 
to use money-handling skills because of a higher incidence of paid jobs. 
(While this difference was not estimated to be significant at commonly 
accepted levels of statistical confidence, this could be the result of the 
previously described estimation deficiences of this preliminary analysis,) 

A smaller proportion of the experimental group received SSI and SSDI 
benefits. However, this does not mean that STETS participation caused a 
decline in the use of these programs; a similar difference was apparent at 
random assignment. (This factor will be controlled statistically in the 
final report when more follow-up data are available.) 

C. Differences in Outcomes by Subgroups 

A thorough examination of outcomes by subgroups requires statistical 
techniques that can isolate the independent effects associated with 
subgroup attributes. This will be done in the final report. The 
discussion here is confined to simple cross-tabulations of outcomes and 
background variables and, consequently , the results are preliminary • 
Additionally, because of small sample sizes and a certain amount of 
interdependence among groups, it is important to remember that trends may 
change considerably as more and longer follow-up data become available. 

Tables 6.4 and 6.5 show the distribution of outcomes by subgroup as 
defined by site, IQ range, sex and age. The results are presented 
separately for the control and experimental groups. 



Demonstration Site . While it is not possible to assess the 




TABLE 6.4 



ACTIVITIES OF CONTROL GROUP AT SIX KONTHS AFTER BASELINE 

BY SUBGROUP 



Subgroup 


Any Paid 

Job 


Regular 
Job 8 


Workshop 
or Activity 
Center Job* 


No Job, 
Training, 
or School 


Financial 
Management 

Skills 


Receipt of 
SSI or SSDI 


Site (X) 














Lot Angeles 
New York 
Cincinnati 
St. Paul 
Tucson 


38.5 
52.9 
41 .7 

Sft ft 

50 .0 


7.7 
17.6 
12.5 

0 .0 
20 .0 


23.1 
8.8 
8.3 
11 .8 
16.7 


26.9 
23.5 
41 .7 
29.4 
36.7 


19.2 
26.5 
25.0 
41.2 
33.3 


26 .9 
24.2 
25.0 
6.3 
56 .7 


Retardation Range (X) 
Border 1 inp 
Mild 

Moderate 


AC Q 

45.1 
57 .1 


22 .4 
6.6 
0.0 


6 .1 
13.1 
33.3 


30.6 
29.5 
38.1 


40 .8 
24.6 
9.5 


18.4 
32.2 
52.4 


Sex (X) 














Male 
Female 


44.7 
52.7 


14.5 
10.9 


13.2 
14.5 


30.3 
32.7 


31.6 
23 .6 


30.7 
29.6 


Age (Z) 














Under 22 Year* 
22 Years and Over 


41 .9 
63.2 


12.9 
13.2 


6.5 
31.6 


35.5 
21.1 


28.0 
28.9 


27.2 
37.8 



SOURCE: Six-Konth follow-up survey of the STETS impact analysis, administered by Mathenatica 
Policy Research from April 1982 through June 1983. y nacneffl " ic « 

NOTES: Sample size is 131. 

t M <.<.. u R T- a w j° bs . include a11 P» id jobs of over 4 hours per week which are not specified as 

5l«S2-f 'r l tl " " W0rk8h0pB or activit y c «^ers. See Appendix C for a complete 
description of how the outcome measures were defined. F 
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TABLE 6.5 



ACTIVITIES OF EXPERIMENTAL GROUP AT SIX MONTHS AFTER BASELINE, 

BY SUBGROUP 



Subgroup 


Anv Paid 

Job 


Regu lar 
Job* 


Uor V ihnn 

tiT Aftl VI tV 

Center Job 


No Toh 
Trii nmo 

or School 


Fi nin ri ll 

n €i i \m jl cm c gi u 

Skills 


SSI or SSDI 


Site (X) 


















17 2 


3 4 


37 9 


37 9 


41 4 


New York 


74.3 


2.9 


0.0 


14.3 


37.1 


11.4 


Cincinnati 


50 .0 


12.5 


16.7 


29.2 


45.8 


12.5 


St. Paul 


73 3 


26 7 


1 3 3 


9fi 7 


&fi 7 


13 3 


*Tll p t on 


93 Q 


6 .1 




3 0 


27 3 


51 5 


IQ Range (X) 














Borderl ine 


68 8 


18.8 


2 .1 


20 .8 


47 .9 


12.5 


Mill! 




8 8 


1 0 3 


50 6 


33 A 


32 A 


Moderate 


65.0 


0.0 


5.0 


20 .0 


25.0 


50 .0 


Sex (Z) 














Male 


69.9 


17 .8 


9.6 


16.4 


41.1 


26 .0 


Female 


66.7 


3.2 


3.2 


25,4 


33.3 


30.2 


Age (Z) 














Under 22 Years 


65.8 


11.4 


2.5 


19.0 


34.2 


17.7 


22 Years and Over 


71.9 


10.5 


12.3 


22.8 


42.1 


42.1 


SOURCE: Six-Month follow-up 


survey of 


the STETS impact analysis 


, administered by 



Mathematics Policy Research from April 1982 through June 1983. 



NOTES: Sample size is 136. 

^Regular jobs include ail paid jobs of over 4 hours per week which are not 
specified as training or school jobs or in workshops or activity centers. See Appendix C for a 
complete description of how the outcome measures were defined, 
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relative impact of each site with the data currently available, it is 
notable that interaite variations on several outcome measures are 
considerable within both the experimental and control groups. Among 
controls y for example, the proportion of respondents who were not in a job, 
training program or school at the six-month point ranged from a low of 24 
percent in New York to a high of 42 percent in Cincinnati. The proportion 
of such clients in the experimental group covered a much wider range: from 
3 percent in Tucson to 38 percent in Los Angeles. It is interesting to 
note that Los Angeles was the only site where more experimentals than 
controls were inactive at the six-month point. 

The proportion of respondents holding a regular job also varied 
considerably by site. Within the control group, none of the respondents in 
St. Paul held such positions, while, at the other extreme, 20 percent of 
those in Tucson were in regular jobs. Large variations similarly existed 
within the experimental group. However, since many experimentals were 
still enrolled in the STETS program, it cannot be concluded that the sites 
with smaller proportions of regular job holders were performing less well 
than the others. 

2. Level of Retardation . In the control group, the level of re- 
tardation was related to the kind of paid job that a person held. Members 
in the borderline range were most likely to be in regular jobs and least 
likely to work in sheltered workshop positions; those in the mild range 
usually held training jobs rather than regular and workshop jobs; and those 
in the moderate range were most likely to have workshop jobs and not 
regular jobs. Moderately retarded persons were also most likely to be 
inactive; 38 percent reported no job, training or school activity. 
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In contrast , patterns of labor market activity in the experimental 
group were quite similar across retardation categories. From 65 to 69 
percent of respondents in each category held a paid job. Similarly, about 
20 percent of the exper imentals in each range were inactive. 

The absence of such sharp distinctions in the experimental group may 
reflect the more even treatment provided by the STETS program to members 
with different levels of retardation, a practice not so evident in the ser- 
vices received by control group members. This evenness may also have con- 
tributed to the lover variation in financial management skills across retar- 
dation levels of the experimental group. Among controls, these skills 
varied more widely, in large part because the proportion of moderately 
retarded persons demonstrating such skills was so low (10 percent). 

Finally, within both samples, there was a strong negative correlation 
between retardation level and receipt of SSI and SSDI benefits. As a rule, 
the relationship is inherent in the eligibility criteria of the two pro- 
grams and is unlikely to be affected by only six months of STETS partici- 
pation. 

3. Sex . On the whole, sex-based differences in activities at 
six months appeared to be greater for the experimental group than for the 
control group. For e^aaple, in contrast to males in the control group, 
males in the experimental group were notably more likely than females to 
have been in regular or workshop jobs, and they were less likely to have 
reported not being in any job, school or training activity. 

This pattern is reversed, however, for the "any paid job" variable. 
Here the control group registered a greater variation between sexes, with 
53 percent of the females, compared to 45 percent of the males, reporting 
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such positions* Unfortunately, the reasons for these patterns are not 
clear at this point. However , they do suggest that an important question 
in the final impact analysis will be whether STETS was more effective for 
one sex than the other, 

4. Age , Age was another variable related to activity patterns 
at the time of' the six-month interview* Like level of retardation, it was 
more strongly related to the probability of controls being inactive than 
was true for the experimental group* Among controls. 36 percent of 
respondents who were under 22 years of age at the time of the follow-up 
interview reported having no job, training program or schooling activity , 
compared to only 21 percent of those who were age 22 or older* Among 
experimental, however, the older group was more likely to be inactive 
compared to the younger group, although this difference was not large (23 
versus 19 percent)* It is also interesting to note that, among controls, 
the greater activity of the older group is largely accounted for by their 
higher rates of participation in workshops and activity centers. 

There was also a positive correlation between age and the receipt of 
SSI and SSDI benefits* Older control group members were more likely than 
younger members to receive these benefits (38 percent versus 27 percent)* 
This dependence was much more apparent in the experimental sample, with 42 
percent of the older group receiving payments compared to just 18 percent 
of the younger group* 

It is important to re-emphasize the preliminary nature of these 
findings* Some of the trends observed here will be explored further in the 
final impact re'port, where a larger sample, longer follow-up and more 
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sophisticated statistical techniques will allow for more complete and 
precise estimates to be nade of the effectiveness of STETS. 
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CHAPTER VII 
PROGRAM OPERATING COSTS 



In a judgment about the effectiveness of STETS, the costs of the 
program and how they compare with the benefits are important factors. The 
program's evaluation plan therefore incorporates a comprehensive 
benefit-cost analysis, with results to be presented, along vith the impact 
analysis, in a later report. At this point, however, it is possible only 
to make some initial estimated oJ costs, and to present some preliminary 
findings from one component of w ut analysis, in which an assessment is 
made of the value of output (gooU. and services) produced by participants 
while they were still in the program. These, along with the framework of 
the benefit-cost analysis, are the concerns of the present chapter. 

I. Site Operating Costs 

The concepts, assumptions and data on which cost estimates are based 
are discussed first in this section. Cost data were reported by the local 
sites on a monthly basis, using standardited fiscal forms designed by MDRC. 
These forms required that program expenses be separated into several major 
categories : 

Participant Compensation ^ which refers to payments made to 
participants while they were in Phase 1 or 2. 

Program Management , which is a very broad category that 
encompasses general operational overhead, program planning, 
job development, information system management and fiscal 
accounting. 

Worksite and Skills Training , which refers to the costs 
incurred by the program in providing job training and worksite 
supervision to STETS clients. (This does not include any of 
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the costs borne by the outside employer in providing such 
services.) 

Client Services , which encompass a variety of support services 
intended to help clients overcome a range of obstacles to 
employment. Such services included individual and group 
counseling, travel training, job matching, and vocational 
needs assessment. Most of these were provided directly by the 
program, but some were purchased from other local agencies. 

In practice, the distinctions between the last two categories proved 
to be somewhat ot'-ure. Sites often had difficulty separating training 
from counseling services when reporting expenses, largely because training 
strategies generally entailed counseling on work-related problems and 
performance, often at the worksite. The New York site, for example, 
reported rero expenses under the client services category, even though 
work-related counseling was provided frequently. Consequently, for the 
discussion in this chapter, the expenses reported in the two categories of 
"training" and "client services" have been combined. 

Sites wers also required to report funding sources, including local 
and national grants from government and private agencies, contributions 
from their sponsoring agencies, and "service project revenue," which refers 
to income generated by participants during their tenure in the program. In 
those sites using r.he sheltered model for Phase 1, service project revenue 
encompassed all receipts from the sale of goods and services produced by 
STETS participants in their workshop jobs. For sites using the Phase 1 
non-sheltered model and for all sites in Phase 2, it covered any employer 
reimbursements made to STETS in return for participants' labor. In 
addition to these sources of revenue, all sites reported any wages paid 
directly to participants by employers. 

An important aspect of the cost analysis is its focus on a segment of 
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calendar tine — the •'Ateady-state" period — within the overall demon- 
stration period. The steady-state period is that segment of tine during 
which program operations most reflected the intensity, scale and efficiency 
that could be expected during a stable period of operations in a replicated 
program. Because atart-up costs for any program are usually much higher 
than the costs of ongoing operations, estimates of normal operating costs 
would be distorted if those early expenses were counted. Similarly, the 
costs of closing a program down are likely to be atypical. Hence, only the 
costs incurred during the steady-state period are analyzed here. 

In theory, a steady-state period begins at the end of start-up and 
continues until the beginning of phase-down. In STETS, however, the 
relatively short time of the demonstration constrained sites from ever 
reaching a mature state of operations. Within approximately eight months 
after random assignment began, intake was sharply curtailed because the 
sites had been informed that national funding could not be guaranteed past 
December 1982. While funds eventually did become available to continue the 
project for another year, activities had to be phased down over that period 
of time. Consequently, it is difficult to identify a "true 11 steady-state 
period for STETS. (It is instructive to note that, in the National 
Supported Work Demonstration, the third year of program operations was 
defined as the steady-state period, while the first two years were viewed 
as start-up. See Kemper, Long and Thornton, 1981.) 

Thus, while a "true" steady-state period cannot be- identified for 
STETS, it is possible to specify several months for each site during which 
activities were fairly stable and a relatively high number of participants 
were actively served: March-July 1982 for New York, July-November 1982 for 
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Lot Angeles, May-September 1982 for Cincinnati, June-October 1982 for St. 
Paul, and May-September 1982 for Tucson. 1 For the cost calculations 
presented in this chapter, these .periods, taken together, are defined as 
the steady-state period for the demonstration as a whole. While these 
periods do not represent a mature phase of operations for any site, they do 
at least exclude the months during which start-up problems were greatest 
and those when the curtailment of program services was fully under way. 

Costs have been estimated in terms of several different units to 
provide alternative perspectives on the level of resources required to 
operate the program. For each of these measures, the calculations use data 
from the steady-state period cnly, and focus on the time that part :ipanta 
were enrolled in STETS. 2 However, the estimates do not include expenses 
incurred by the sites for research-related activities that would not 
normally be undertaken,"* nor do they include the central administrative 
costs incurred by MDRC to monitor and assist the sites. No adjustments 
were made for inflation or for differences in general salary or price 
levels across the demonstration cities* 

The following types of unit costs have been estimated. Net Cost Per 
Service Year represents the cost of serving one participant for one year, 
regardless of how long participants, on average, actually stayed in the 
program. This figure excludes service project revenue (that is, program 
income from participants' labor). The focus on a year of service makes it 
possible to estimate the likely expense of operating a STETS program for 
one year with a specified number of slots. Net Cost Per Participant is the 
average cost of serving an individual participant. This estimate is 
adjusted for the average length of time that participants were enrolled in 
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the program. 

Table 7.1 shows the estimates for each of these unit costs for the 
steady-state period* For the demonstration as a whole, the net operating 
cost per service year was $8,715* Because most participants did not stay 
in the program for a full year, the average cost of serving each client, 
$7 ,553 , was lower. 

In addition, a second set of estimates — again, one per participant 
and another per service year — was made* These estimates add to the 
previous cost figures both the wages paid directly by employers to 
participants and service project revenue* They illustrate what the costs 
of the program would have been if the program had not been able to find any 
employers to pay participants and if participants had not produced goods 
and services with value* Table 7*1 shows that the average cost of the 
program per year under these circumstances would have increased by 15 
percent (from $8,715 to $10,021).^ Similarly, the cost per participant 
would have increased by 15 percent* These calculations demonstrate the 
utility of the program's generating employer payments and service project 
revenue as a way of containing costs* 

To put the costs into some perspective, it is useful to compare them 
to the costs of other employment programs for retarded persons. This is 
difficult, however, for the same reason that it was difficult to compare 
placement rates across programs. (See Chapter V«) Because published cost 
data, like placement data, are scarce and because the methodologies used to 
calculate costs vary across programs, comparisons with other programs must 
be made with caution* 

With these limitations in mind, the costs of several other employment 
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TABLE 7.1 



AVERAGE UNIT COSTS OF OPERATING STETS DURING 
THE STEADY-STATE PERIOD, BY PROGRAM SITE 



Site 


Net Cost Per 
Service Year 


Net Cost Per 
Participant 


Net Cost Plus 
SPR and Employer 
Payments, Per 
Service Year 


Net Cost Plus 
SPR and Employer 
Payments, Per 
Participant 


Lot Angeles 


$8,743 


$7,286 


$10,387 


$8,656 


New York 


11,467 


9,651 


11,778 


9,913 


Cincinnati 


10,311 


8,420 


12 169 


Q ft 


St. Paul 


5,411 


4,283 


7,604 


6,020 


Tucson 


6,724 


7 ,060 


7,708 


8,0 94 


All Sites 


8,715 


7,553 


10,021 


8,685 



SOURCE: MDRC tabulations of sites' monthly Combined Operating Reports. 

NOTES: Site costs are based on five months of steady-state operations 
during 1982. Actual dates vary by site. Research-related expenses are not 
included . 

*Net cost per service year is the average cost of serving one 
participant for one year, net of n&rvice project revenue. 

b 

Net cost per participant is the average cost of serving one 
participant for the average length of enrollment in the program, net of service 
project revenue. 

"SPR" refers to service project revenue. 

d "Employer payments" are vages paid directly to participants by 

employers . 
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and training programs can be used as a rough guide for assessing the 
relative magnitude of the STETS costs. As Table 7.2 shows, the service 
year cost of STETS appears to be somewhat higher than the costs incurred by 
sheltered workshops serving mildly and moderately retarded persons. 
However y the cost of STETS per participant may be lower than that of many 
workshops because the average participant time in the STETS program is 
shorter. (See Chapters III and V.) 

The STETS service year cost was lower than a year in the Job Corps, an 
intensive residential employment program for disadvantaged youths which, in 
an evaluation conducted by Mathematics Policy Research (Mallar et al., 
1982), shoved a positive benefit-cost outcome. When per participant costs 
are considered, however, Job Corps and STETS are roughly comparable. 
Finally, the STETS service year and per participant costs appear to be 
close to those of other transitional employment programs for the mentally 
retarded. 

Within the STETS demonstration, the cost of operating the program 
varied substantially across the sites (Table 7.1). New York and Cincinnati 
were the most expensive programs (net costs per service year were $11,467 
and $10,311, respectively), while Tucson and St. Paul cost the least to run 
(net costs per service year of $6,724 and $5,411). Nevertheless, it is 
interesting to note that one major difference across sites — the uce of a 
sheltered versus a non-sheltered Phase 1 approach — did not consistently 
predict the level of costs. For example, Cincinnati's net cost per service 
year was fairly close to New York's.^ 

The lover costs of the St. Paul and Tucson projects are particularly 
notable. In St. Paul, this resulted in part from a fairly high level of 
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TABLE 7.2 



ESTIMATED AVERAGE OPERATING COSTS FOR SELECTED 
EMPLOYMENT AND TRAINING PROGRAMS 
(FISCAL YEAR 1982 DOLLARS) 



Program 


Average Cost 
Per Service Year 


Average Cost 
Per Participant 


ctftc 1 

STLT5 


$8,715 


$7,550 


Sheltered Workshops 2 ^ 


4,000-6,000 


c 


Project Employability ' 


c 


6,000 


Employment Training Program^** 


8,170 


c 


University of Washington 
Food Service Project ' 


10,960 


9,750 


Job Corps 6/d 


15,580 


7,920 



SOURCES: 1. Table 7.1. 

2. Personal communication with officials in public agencies 
in several STETS sites. 

3. Mark Hill and Paul Wehman. "Cost Benefit Analysis of 
Placing Moderately and Severely Handicapped Individuals into Competitive 
Employment." The Journal of the Association for the Severely Handicapped 
Vol. 8, 1983, pp. 30-38. ~~ ^~ 9 

4. Ken Schneider, Frank Rusch, Robert Henderson and Terry 
Geske. "Competitive Employment for Mentally Retarded Persons: Costs Versus 
Benefits." Unpublished paper, University of Illinois at Urbana-Champaign. 
1982. 6 ' 



5. James Moss. "Poot secondary Vocational Education for 
Mentally Retarded Adults." Reston, VA: ERIC Clearinghouse on Handicapped 
and Gifted Children, 1980. 

6* Robert Taggart. A Fisherman's Guide: An Assessment of 
Training and Reme diation Strategies . Kalamazoo, MI: UpJohn Institute for 
Employment Research, 1981. 

NOTES : Project Employability is a transitional employment program 
for mildly, moderately and severely retarded persons operated out of 
Virginia Commonwealth University* The Employment Training Project is a 

(continued) 
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TABLE 7.2 (continued) 



transitional employment program for mentally retarded persons operated out 
of the University of Illinois, Urbana-Champaign. The University of 
Washington Food Service Project is an employment program to prepare mildly 
and moderately retarded persons for jobs in the food service field. Job 
Corps is an intensive residential training program for disadvantaged 
youths. 

Different methodologies were used in estimating the costs 
for these programs. Caution should therefore be exercised in comparing 
programs. 

Estimates are expressed in Fiscal Year 1982 dollars. 
Adjustments have been made using the change in the implicit price deflator 
for gross national product. 

*STETS costs are net of service project revenue. 

Estimates are based on costs for serving individuals in the 
mild to moderate range of retardation. Costs may vary considerably 
depending on the locality of the workshop and the level of services 
provided. 

c Not available. 
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direct employer payments to participants. Because most Phase 2 
participants in these sites were in unsubsidised training positions, the 
total amount of participant compensation that had to be paid by the program 
vas lessened. Additionally, during phase 1, St. Paul (in contrast to the 
other sites) vas allowed to pay piece rates to participants. These wages, 
between $1 and $2 per hour, resulted in savings of roughly $500 to $1,000 
per participant. Also relevsnt was the understaffed job development effort 
during much of the demonstration period, which reduced talary expenses. 

The reasons for the relatively lower net cost per service year in 
Tucson are less clear but may, to an important extent, reflect lower staff 
salaries at that site. It should be noted, however, that Tucson's per 
participant cost is close to the average for all sites. It appears that 
lower staff salaries and other fiscal advantages in Tucson were offset by a 
higher average length of participant stay. 

Table 7.3 shows the allocation of net operating costs across the three 
major cost categories of the sites' budgets. The largest share (37 
percent) is accounted for by "program management and job development," but 
roughly equal amounts were spent on "client training and services" and 
"participant compensation" (31 and 32 percent, respectively). It is 
important to stress that the broad definitions of the first two categories 
make it difficult to isolate precisely the level of resources devoted to 
client training and support services. Many staff activities encompassed by 
program management and job development could by other definitions be viewed 
as training efforts. For example, high-level staff members, in addition to 
their administrative and management responsibilities, often assisted other 
staff members in providing direct services — helping to counsel clients, 
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TABLE 7.3 



PERCENTAGE DISTRIBUTION OF NET OPERATING COSTS 
DURING THE STEADY-STATE PERIOD, 
BY BUDGET CATEGORY AND PROGRAM SITE 



Site 


Program 
Management and 

Job Development 


Client Training 
and Services 


Participant 
Compensation 


Total 


Lot Angeles 


36.8 


Z5.5 


37.6 


100.0 


Nev York 


43.5 


18.2 


38.3 


100.0 


Cincinnati 


22.1 


50.7 


27.2 


100.0 


St. Paul 


45.3 


37.8 


16.9 


100.0 


Tucson 


37.6 


35.1 


27 .3 


100.0 


All Sites 


36.8 


31.3 


32.0 


100.0 



SOURCE: MDRC tabulations of sites' monthly Combined Operating Reports. 

duri«r?f« !\ tC ?°! t * b "! d 0n Hve BOnth, of "e-dy-state operations 
during 1982. Actual dates vsry by site. Costs are net of service project 
revenue and exclude resesrch-related expenses. 
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assess their employment difficulties, design their training pia. * and 
select appropriate job placements. 

This blurring of functions is an important issue, given the recent 
thrust of federal funding for employment and training programs. The Job 
Training Partnership Act (JTPA), which is currently the dominant source of 
national funding for employment and training activities and potentially a 
major resource for future STETS efforts, generally requires that 70 percent 
of JTPA funds be allocated to "training" (as this category is defined in 
the legislation) and that only 30 percent be spent on administration and 
support services. However, the definition of "training" in the JTPA 
regulations would include a portion of the expenses in the STETS category 
of "program management and job development." Thus, it should be possible 
to replicate STETS using JTPA funds, but probably only in conjunction with 
revenues from other funding sources. 

II, Analytical Framework for the Benefit- Cost Analysis 

As mentioned earlier, Mathematica Policy Research is conducting a 
study that will examine how the costs of STETS compare with the economic 
benefits it generates. It is only possible at this point to describe some 
of the main features of the analysis and to provide estimates on one 
important component: the value of output. 

In general, benefit-cost analyses address two types of questions about 
program evaluation: Is the program economically efficient? Is it equi- 
table? The question of efficiency concerns the effect of a program on the 
total value of the goods and services available to society. The equity 
issue concerns the distribution of goods and services among groups in 



9 

ERJC 



-136- 

160 



society, how the program affectt that distribution, and whether a specific 
group benefits or loses* Benefit-cost analyses do not, however, provide a 
value judgment on whether any particular change in the distribution of 
resources is desirable; they can only describe the nature and extent of the 
redistribution* 

Further, some of the benefits produced by the program may be intan- 
gible or difficult to measure. For example, as participants become 
competitively employed and part of the mainstream, the program may lead to 
improvements in their psychological well-being and perhaps that of other 
members of their families. While such benefits are difficult to quantify, 
they should be taken into consideration in making decisions about program 
replication. 

The benefit-cost analysis is also limited by the inherent difficulty 
of projecting the effects of STETS in economic circumstances different from 
those at the time of the study and under different program designs with 
varying implementation strategies. Nevertheless, the findings from this 
analysis will provide much useful information for assessing the merits of 
STETS . 

The assessment by Mathematica Policy Research of program benefits and 
costs will be carried out from three perspectives: that of participants, 
that of other members of society (whose taxes pay for the program) , and 
that of both groups combined. From the participant perspective, it is 
important to know whether any increase in employment, earnings and 
independence outweighs the potential loss of transfer benefits (such as 
SSI) and exceeds any other costs incurred because of participation. From 
the non-participant perspective, it is important to determine whether the 
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taxes eventually paid by participants and any reduction in their use of 
other publicly supported services are of sufficient magnitude to preclude 
the peed for a net taxpayer subsidy to the program. Finally, from the 
social perspective, it is important to assess the program's overall effect 
on the value of the goods and services available to society as a whole, ^ 

The potential benefits of STETS can be grouped into five categories: 
output produced by participants after they left STETS; participants' 
reduced use of publicly-funded programs other than STETS ; participants 
moving to lover-cost and more independent residential situations; 
participants' reduced reliance on transfer programs; and other (unmeasured) 
benefits. 

Site costs can be grouped into four categories: operations, central 
administration, participant labor, and work-related expenses. An addi- 
tional program-related expense is the cost of central oversight — i.e., 
the activities necessary to administer the contracts with STETS sites and 
to monitor the program at the demonstration-wide level. In the demonstra- 
tion, MDRC performed these functions. In an ongoing program, these costs 
would be incurred by the state or federal agencies responsible for funding 
the local programs, for example, a state Department of Vocational 
Rehabilitation purchasing STETS-type services from a site would incur 
monitoring costs. Alternatively, if a program were funded directly by 
state or federal agencies, such as CETA or JTPA, the central funding 
agencies would incur these costs. 

III. The Value of In-Program Output 

The value of goods and servicer produced by participants in an 
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employment program can substantially offset its costs. One measure of the 

value of this output is the combined value of service project revenue and 

wages paid directly by employers to participants. Measuring this revenue 

provides a reasonable estimate of the value of in-program output 9 and 

because it can be derived with relative ease, it can be incorporated into 

estimates of net operating costs when budgets are planned or evaluated. 

However, the resulting estimate is likely to fall in the lower part of 

a range. Presumably, employers will not pay more for a product (partici- 

8 

pant labor service) than it is worth to them; they may, however, pay less. 
This latter possibility might have beld true in STETS, partly because the 
sites, in negotiating with employers, did not focus on maximizing 
participants' wages beyond the minimum wage level; rather, their priority 
was employment. 

An alternative method for estimating the value of in-program output is 
to use data collected in a series of work activity case studies. For 
STETS, such studies were conducted on the output produced by 40 randomly 

selected participants. For each person, two estimates were made of the 

q 

value of the output they produced during a two-week reference period. One 
estimate was based on the alternative supplier's price of the participant 
output — that is, the alternative labor cost of producing that output. 
This estimate assumes that employers could obtain additional labor at the 
wages paid to their regular employees, and it ignores all additional costs 
that may have been imposed on the employers by the use of the labor of 
STETS participants. The second estimate attempts to deduct these 
additional costs. It estimates the net value of the participant to the 
employer and subtracts the costs of extra employer-provided supervision and 



-139- 



ERLC 




the reduced output from other workers, which arc both factors that may 
result from using STETS labor. 

The net value differs from the supply price for a number of reasons. 
The participant's disability may require supervision or training. For 
example, one firm using a participant to clean its offices received very 
little net valut, even though the participant performed good work at a 
productive pace; that firm provided considerable training and was unwilling 
to scale back its purchase of alternative cleaning services until it could 
dispense with these services altogether. In another case, the clerical 
staff "donated" their simplest jobs to the participant — photocopying, 
mailing and answering the telephone during employee breaks. The work 
performed by this participant was both useful and necessary, but resulted 
in reduced efforts by other employees. 

Both the supply price and the net value of participant output esti- 
mates are useful indices of the value of in-program output. They are baaed 
on careful, systematic interviews conducted in person with participants' 
supervisors. Whenever possible, actual wage, fringe benefit and production 
records were consulted. Given an estimated average length of active 
participation of five months in Phase 1 and six months in Phase 2, the 
supply price estimates indicate that, on average, participants produced 
output worth approximately $1,400 during active Phase 1 participation and 
$3,800 during active Phase 2 time. If the net value of participant output 
estimate is used, the Phase 1 value would be approximately $1,200 during 
active participation, and the Phase 2 value would be $2,700. 

These estimates indicate that participants' in-program output was an 
important offset to program costs. Moreover, the Phase 2 estimates lead 
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credence to the fairly positive ratings of participants' work performance 
measured by the employer survey. (See Chapter V.) Both sett of findings 
suggest that, on the whole, the productivity of STETS participants was not 
inconsequential. 
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CHAPTER VIII 
LESSONS FROM THE IMPLEMENTATION OF STETS 



This report hat described and assessed the ways in which five 
demonstration sites implemented the STETS demonstration. It has also 
examined the costs of operating STETS and some of the ear 1 .; findings from 
the impact analysis. 

It was noted at the outset that the demonstration lasted for a 
relatively brief period , much of it marked by funding uncertainty. 
Nevertheless, a good deal has been learned about the program, and many of 
these lessons should be useful to policymakers and program planners as they 
decide whether or how to replicate the STETS approach. 

The most general conclusion of this report is that the model is a 
feasible one and can be operated on a fairly large scale by Afferent types 
of organizations using diverse approaches. The sites have shown that, with 
varying degrees of success, they could recruit participants, develop 
training positions for them in real work settings, provide them with 
helpful support services, and channel many of them into unsubsidized jobs. 
Given the difficulties faced by the target population, the 
demonstration-wide placement rate of about 42 psrcent is an impressive 
outcome. During the demonstration, the average public cost of operating 
STETS was $8,715 per service year and $7,553 per participant. 

Because of the population it serves, operating STETS can pose a 
substantial implementation challenge. This chapter focuses on specific 
lessons learned as sites attempted to meet that challenge. 
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I. Recruitment and Referral 

The sites' recruitment experiences reveal that the program — and the 
prospect of competitive employment — generally had a strong appeal for the 
population of retarded persons it was intended to serve* Moreover, other 
organizations serving the mentally retarded found a similar promise in 
STETS. Despite random assignment and the other research demands of the 
demonstration, many of these organizations were willing to refer clients to 
the program* 

Within the eligibility guidelines, the sites were able to recruit a 
group of individuals who were fairly diverse in sex, race, presence of 
secondary handicaps, and receipt of SSI and SSDI benefits. Across the 
demonstration, 50 percent of enrollees were classified as mildly retarded, 
while 38 percent fell in the borderline range. Thirteen percent were 
moderately retarded. 

There were several reasons for the smaller proportion of 
lover-functioning participants. First, the fact that fever people in the 
general population are moderately retarded than are mildly or borderline 
retarded meant that proportionately fever vere available for referral. It 
also appeared that some agencies hesitated to refer more severely retarded 
persons before seeing hov veil their higher-functioning clients vould fare 
in the program. Some lover-functioning individuals may have also been 
deterred from participating because of a possible loss of SSI or SSDI 
benefits. 

Tucson and Cincinnati vere fairly successful in enrolling moderately 
retarded persons. This vas facilitated in part by their previous strong 
connections to other agencies serving lover-functioning persons. 



-143- 




ERLC 



Elsewhere, these linkages had to be newly formed, or at least more fully 
developed — not an easy task, given the short duration of the 
demonstration. 

Although the program enrolled a sizable number of participants 
overall, the referral problems at two sites in the early part of the 
demonstration, and the difficulty in recruiting lower-functioning persons, 
attest to the challenge of recruitment. Careful and sustained effort was 
required to acquaint referral agencies with the program and to persuade 
them of its potential benefits. 

In-person presentations to referral agents were fundamental to success- 
ful recruitment* Sometimes these were supplemented by brochures and other 
written descriptions. Furthermore, to maintain a regular flow of refer- 
rals, it was useful to keep agencies well informed of the progress of 
clients they had already referred* 

While it is unknown to what extent SSI and SSDI recipients were 
reluctant to enroll in the program, the complexity of the SSI and SSDI 
regulations do have the potential to pose an important deterrent to 
competitive employment. At minimum, staff members of future STETS projects 
should try to help referral agencies and potential participants, and/or 
their parents or guardians, to understand clearly the rules of these 
disability programs. 

T T The Appropriateness of Different Types of Participants for STETS 

Because sites were able to enroll a diverse group of individuals, it 
may be possible, after the impact analysis is completed, to determine what 
types of mentally retarded individuals are most appropriate for the STETS 
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program. This it an important question, for it relates to the efficient 
use of resources by future programs. To the extent that it is possible to 
identify which types of individuals are likely to benefit, efforts can be 
concentrated on them, while alternative strategies can be devised to help 
persons vho stand less chance of being helped by the program. 

To the extent possible, the impact analysis will address the issue of 
who is appropriate for STETS by comparing the post-program experiences of 
various subgroups of enrollees with the experiences of similar subgroups in 
the control sample. The implementation analysis, while lacking control 
group results, has also addressed this issue, although in a more limited 
way. Relying on in-program performance measures and staff observations, it 
has assessed how well various kinds of participants were able to handle the 
program's demands while they were still enrolled. 

The findings from the implementation analysis indicate that the types 
of individuals who can or cannot adequately meet the standards are not 
clearly identifiable on the basis of objective background characteristics. 
This conclusion is supported in part by the employer survey, which found 
that various demographic characteristics of Phase 2 participants were not 
strongly associated with the performance ratings they earned from their 
supervisors. Additionally, placement rates for those subgroups did not 
vary dramatically from the demonstration-wide average. Consistent with 
these findings, program staff members reported in interviews with MDRC 
researchers that it was difficult to predict at intake who would or would 
not do veil in the program. In part, this was because a large proportion 
of participants who did not do wel 1 had emotional or behavioral problems 
that were not readily apparent in the intake interviews or from the 
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information provided by referral agencies. Staff members reported, 
however, that often they could better judge appropriateness fairly toon 
after enrollment, when they had had an opportunity to observe participants 
working for a short period of time. (This is consistent with the finding 
that most terminations occurred during Phase 1 of the program.) 

This analysis suggests that, on the basis of current data, future 
operators ought not to adopt rigid selection criteria for choosing 
participants. However, program planners might want to consider ways of 
incorporating a short "try-out" period, d^ ing which staff members could 
observe applicants, 

III. Participants' Work Performance 

During their program tenure, participants manifested a variety of 
work-related problems, including deficies es in both work habits and 
skills. Common problems with work habits included inadequate attendance 
and punctuality; inappropriate interaction with supervisors and co-workers; 
inadequate concentration and endurance; difficulties in initiating, 
completing and changing tasks; and improper grooming and hygiene. Among 
work skills deficiencies were inadequate speed; poor reading and math; 
difficulties in understanding or remembering how to do certain tasks ; 
forgetting to complete all tasks assigned; and inability to work according 
to a preset schedule. Of course, all participants did not have all of 
these deficiencies but, taken together, they form the core set of problems 
on which staff members focused their training efforts. 

Previous research, as noted in Chapter I, suggests that retarded 
persons — especially the mildly retarded — more typically have trouble 
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obtaining and keeping jobs because of poor work habits than because of 
unacceptable skills. The findings from this study support that 
interpretation. Staff observations, as well as MIS data, indicate that 
negative terminations were caused much less often by an inability to 
perform tasks than by behavioral and other problems. 

Skill deficits were not insignificant, however. They accounted for 
over 12 percent of terminations. Furthermore, even among the more select 
group of participants who were evaluated in the employer survey, a 
substantial proportion were rated as performing less well in the amount and 
quality of their work than other workers who had been in similar jobs for 
about the same amount of time. 

Thus, while deficiencies in work habits and skxlls were by no means so 
pervasive as to prevent all STETS participants from eventually entering 
unsubsidized jobs, they were common enough to signal a real need for work 
experience, training assistance and other support services. This finding 
supports the program's attempt to provide such experiences and services. 
It also lends credibility to another fundamental concept of STETS: its 
gradual increase in the demands and responsibilities imposed on 
participants ,who need time to adapt to the requisites of competitive 
employment. 

IV. Work Experience and Training 

The experiences of the sites suggest a number of lessons about the 
feasibility, and the advantages and disadvantages, of various strategies 
for preparing participants for competitive employment. 
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A. The Shelt ered Versus Non-Sheltered Approach 

Phase 1 can be operated in both sheltered environment! and 
non-sheltered public or nonprofit agencies , but the choice of approach 
involves important trade-offs. First, the sheltered approach reduces the 
program's job development burden in Phase 1, since it is not necessary to 
find outside training jobs. Second, congregating participants in the same 
work setting (or in just a few settings, as in Tucson), eases counselors' 
travel burdens, allowing them to more easily monitor performance, instruct 
participants and intervene with special assistance. Use of a sheltered 
setting also probably allows a program to be less restrictive in selecting 
enrollees. Because they are not placed immediately into real work 
settings, the program's relationships with local employers are not jeopar- 
dised if workers prove to be unsuitable for the program at this point. 

The non-sheltered model, however, has some key advantages over the 
sheltered approach. One is that Phase 1 jobs are in r«al world settings 
and thus more closely resemble Phase 2 and 3 jobs, which are likely to be 
service-oriented. (Unlike the jobs in subsequent phases, sheltered jobs 
tend to be assembly and packaging tasks.) Hence, non-sheltered Phase 1 
participants may find it easier to make the adjustment to Phase 2 and then 
to regular employment. It may be possible, however, to create more 
service-oriented positions — such as maintenance, messenger and clerical 
positions — within sheltered settings. 

Also, however, certain participants perform better in a more normal 
work environment . Some resent a sheltered environment, particularly if 
they are sensitive about being identified as mentally retarded or 
handicapped. For such individuals, as the Cincinnatti staff in particular 
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observed, the sheltered workshop may be "pert of the problem" end may mask 
the participant's real potential for competitive work. This suggests that 
future STETS programs using this sheltered model should try not to 
evaluate a participant's capacity for a non-sheltered position solely on 
the basis of Phase 1 performance. Operators might also try to expedite 
Phase 2 placements for those whom they believe might perform better in a 
non-sheltered environment. 

Another problem with the sheltered model is that non-STETS workers, 
who usually receive piece rates for their work, may resent program workers 
if they are paid the minimum wage, as in this demonstration. Cincinnati's 
experience, however, suggests that this problem is not necessarily an 
unmanageable one. 

The non-sheltered approach places responsibility for monitoring 
participants on their supervisors at the public and nonprofit agencies. 
The quality of these Phase 1 experiences thus depends largely on the 
quality of support and assistance that those supervisors are able and 
willing to provide. It is critical for programs using the non-sheltered 
approach to select Phase 1 worksites carefully, and having selected them, 
to work closely with the supervisors. 

B. Assessment Strategies 

Phase 1 was designed not only as a low-stress introduction to the 
world of work, but also as a period for careful staff assessment of 
participants' employability problems and service needs. All of the sites 
chose to emphasize observational techniques ~ careful examination of 
participants' behavior and skills as they performed their Phase 1 jobs. 
Some sites also administered formal testing, but this came to be viewed as 
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»t most «n occasionally useful, but by no meant essential, complement to 
the observation!. Indeed, St. Paul staff members concluded that the 
extensive formal evaluations they had required prior to Phase 1 work 
assignment were not useful at all. 

All sites continued to observe participants in the subsequent phases 
during their worksite visits. In fact, training and assessment went 
virtually hand-in-hand throughout the program. 

c « The 500-H our Rule «nd Other Time Limits 

In an effort to test a relatively quick transition to competitive 
employment, the program model imposed several constraints on length of 
stay. The requirement that participants be limited to 12 months of paid 
participation within a 15-month period was not found to be an unreasonable 
burden. Participants were enrolled in STETS for an average of about 10 
months. 

The 500-hour limitation on paid activities in Phase 1 was a more 
troubling guideline, primarily in the early stages of the demonstration. 
Some staff found this time limit insufficient for preparing a number of 
participants for Phase 2 jobs, particularly in the private sector. 
However, for the most part, it was concluded that if individuals were not 
ready for a Phase 2 job within 500 hours, they were also not likely to 
benefit from a modest extension of that limit. While some exceptions may 
be appropriate, the rule is an effective source of pressure on programs to 
pursue Phase 2 and 3 jobs with diligence. 

D« Phase 2 in Real Work Settings 

Operating Phase 2 in real work settings, whether in private sector 
firms or public or nonprofit agencies, is a feasible option. Many 
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employer! were willing to take on Phase 2 participant! even though they 
were still program trainee! . Furthermore, the employer mrvey show* that 
most supervisors of Phase 2 participants did not find either work 
performance or the demands of the program to be unacceptable. The majority 
said that they would again become involved with the program if ssked to do 



so. 



At certain Phase 2 worksites, however, itaff members had to limit 
their monitoring and assistance, particularly in private sector firms where 
the jobs were unsubsidired. Some employers did not like counselors to 
disrupt the work process. One useful strategy was to visit participants 
before they began work or during their lunch breaks. 

Compared to the other sites, New York relied more heavily on 
subsidized public and nonprofit agencies for Phase 2. One advantage of 
this approach was that staff members could more easily monitor and assist 
participants on the job and influence, through informal negotiations with 
the supervisors, the tasks assigned to participants. They could also, to 
some extent, see that participants were rotated through a variety of jobs, 
which would train them in multiple skills. One disadvantage, however, was 
that it was difficult for participants to move directly into Phase 3 jobs 
at the same worksite. 

E « The Importance of Support Services During Phase 3 
The final phase was intended to be a period during which itaff 
provided follow-up iupport services to participants in order to help them 
adjust to their competitive jobs. These services were to be gradually 
reduced over a six-month period, at which point they were to end. The 
sites' experiences reveal that, for most participants who entered Phase 3, 
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this treatment was sufficient to help them adapt to their jobs. 

For seme, however, independent employment created new stresses. Among 
these were the greater demands of Phase 3 employers, changes in job 
descriptions, or a change of employer or supervisor. Thus, several of the 
sites found that they had to provide significant amounts of counseling, and 
sometimes additional training, on an as-needed basis, for a small number of 
Phase 3 participants. While this kind of support was not extensive, it was 
considered important in helping certain participants to remain 
competitively employed. 



V. Job Development 

Job development in both Phase 2 and 3 proved to be a very difficult 
task. This difficulty is reflected in the results of the job development 
contact study, which found that only 4 percent of the employers contacted 
during the study period agreed to provide jobs. Nevertheless, the sites 
were able to recruit sufficient numbers of employers so that it was usually 
not necessary to terminate capable participants simply because jobs were 
unavailable. Yet, overall, the supply of jobs was tight ~ so much so that 
other aspects of the program implementation were hampered. 

For example, the long lead time needed for Phase 2 job development led 
several sites to keep a number of participants in Phase 1 positions longer 
than necessary. Many participants also had to be placed in a "hold" status 
after their allotted 500 paid hours because Phase 2 jobs were not yet 
available. A similar delay sometimes occurred between Phases 2 and 3. The 
limited supply of jobs in both phases also constrained the ability of the 
sites to match particinants to jobs, according to their interests and 
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capabilities, which in some cases may have adversely affected job 
performance. 

Job development was difficult for several reasons, one being the deep 
recession during much of the demonstration period. Another vas that the 
job developers in all sites except Hew York generally lacked sufficient 
experience in finding suitable regular jobs for this population. STETS vas 
also a new program, and the sites had little time in which to refine and 
strengthen their job development techniques, or to establish a positive 
"track record 11 among employers. These difficulties vere compounded by the 
nature of the population. Not only vere participants mentally retarded, 
they vere young and frequently non-vhite, and faced all the labor market 
obstacles associated vith these factors. 

Despite the constraints, many employers sponsored STETS enrol lees. 
The employer survey shows that, among Phase 2 employers, altruism — 
expressed as the desire to help participants or the community — was an 
important reason for taking on STETS trainees. However, while this might 
have encouraged many employers to accept participants as trainees, it did 
not necessarily mean that they would hire them. Participants had to 
demonstrate a reasonable level of proficiency during Phase 2. 

For most employers, factors other than altruism also influenced hiring 
decisions. A significant proportion of Phase 2 employers cited the 
financial subsidies offered by the sites, the promise of assistance in 
training, t v previous work experience of participants in Phase 1, and the 
availability (to private sector firms) of the Targeted Jobs Tax Credit. It 
would thus be useful for operators of future programs to incorporate these 
incentives into their job development efforts. 
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The job development process varied among the sites. A comparison of 



the different approaches and divergent outcomes suggests a number of 
lessons about how this process might be improved. 

• Job development appears to be less effective vhen that task 
is assigned to the same staff members vho are responsible 
for counseling and training. Because job development is 
complicated and time-consuming , it is best considered as a 
specialised function within the program. 

• The emphasis in job development is better placed on 
generating a "bank" of jobs — that is, a supply of jobs 
that might be suitable for any number of clients — rather 
than trying to find jobs tailored to specific individuals. 
In a tight labor market, the latter practice delays 
placement* 

a Guidelines concerning the types of jobs and industries on 
which to target efforts need not be defined rigidly. Within 
realistic limits of participants' capabilities and other 
contingencies, a variety of jobs in a wide range of settings 
can be pursued. However, it is likely to take longer to 
find positions in larger firms because there are more 
procedures to be followed before hiring decisions can be 
made. 

• It would be helpful for job developers to have business 
experience. This qualification if much more important than 
counseling or vocational rehabilitation experience. A 
background in business can help job developers understand 
and address the needs and concerns of employers. 

• It is useful for job developers to consult routinely with 
program counselors and to taeet with participants in order to 
gain a firm understanding of the strengths and weaknesses of 
the types of persons they must try to persuade employers to 
hire. 

• Higher-level staff members in STETS or its sponsoring agency 
can usefully contribute to job development efforts, in part 
by presenting the merits of the program to groups of 
employers through special events and meetings. 

• Advisory boards , composed of local businessmen and perhaps 
union leaders, can be helpful in directing job developers 
toward types of jobs and firms that are likely to provide 
employment opportunities for participants. However, an advi- 
sory board can only complement $ and should not be expected 
to substitute for, the strong efforts of job developers. 
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TO* STETS in the Larter Network of Service! for Mentally Retarded Persons 

Given the basic feasibility of the STETS model, and its capacity to 
•erve individuals with a diverse range of prior employment and training 
experiences, it is important to consider where the program might best fit 
into the larger network of services for mentally retarded persons. 

The implementation analysis shows that the model is a fairly adaptable 
one, and that the program can be administered by a sheltered workshop, a 
state agency, or a nonprofit gaining organization. However, if STETk is 
part of a broader array of programs within a parent agency, it must have 
the strong commitment of high-level administrators in the parent 
organization. Also, as indicated by th<* experience of the Los Angeles 
site, che program is at a disadvantage if it relies on other sponsoring 
agency staff unless STETS has direct authority over their work. Because 
the Los Angeles director did not have that control, participants at that 
site were not given needed attention. 

Another aspect of the larger network affecting program implementation 
is the flexibility of funding arrangements. Because funding agencies 
typically restrict the ways in which their monies can be used, it may be 
difficult to structure a program exactly as the model specifies. Some 
funding agencies, for example, may not pay wage subsidies for on-the-job 
training in private sector firms, or may not support participant wages at 
ell. Others ma> place strict limits on the amount of work experience they 
will fund or the type* of clients that can be served. Implementation is 
thus easiest when funding is flexible or when multiple sources of revenue, 
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without overlapping restrictions, can be obtained. 

While the STETS model appears to have considerable flexibility, this 
research cannot ascertain whether other types of organizations — such as 
schools or vocational rehabilitation agencies — could operate the program 
effectively. Nor can it assess the feasibility and value of many types of 
linkages (beyond simply the referral of clients) between outside 
organizations and STETS. Finally, this research cannot predict the 
capacity of the program to serve mentally disabled individuals other than 
the types of individuals enrolled in this demonstration. It would be 
useful for further research to address these issues. 

It should also be remembered that the implementation issues analyzed 
in this report cannot answer other important questions that are relevant to 
a program replication effort. While the model appears feasible under a 
variety of conditions, it remains to be seen whether it actually leads to 
better post-program social and economic outcomes for participants. These 
issues will be addressed in the demonstration's impact and benefit-cost 
analyses. 
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CHAPTER 1 



1. Research on this issue is limited, and many of the previous 
studies suffer from serious methodological flaws. For example, 
many are very small-scale follow-up studies of retarded persons 
who were formerly in special education classes or a wide variety 
of vocational programs, making it difficult to generalize results 
to the larger population of retarded persons* Additionally, much 
of the research is outdated ; most studies were conducted before 
1970, under different economic circumstances and before many of 
the major legislative changes that expanded the rights of and 
services to handicapped persons* These studies also use different 
measures of employment success; some report on whether a 
respondent was employed at the time of the interview, while others 
focus on whether a person was "usually employed" or "ever 
employed*" Studies are also not clear on the nature, quality or 
stability of employment* For a comprehensive review of these 
studies, see Conley (1973)* 

2. The category of "borderline retarded" was officially eliminated in 
the American Association on Mental Deficiency's 1973 revised 
definition of mental retardation; the classification is, however, 
still widely used. It should also be noted more generally that 
behavioral deficiencies, in addition to low intelligence, are 
critical to the official definition of mental retardation adopted 
by the American Association on Mental Deficiency, which reads as 
follows : "Mental Retardation refers to significantly subaverage 
general intellectual functioning existing concurrently with defi- 
cits in adaptive behavior and manifested during the developmental 
period" (Grossman, 1973:11). 

3* See, for example, L. Hartlage, P. Rolland, and D. Taraba, "Percep- 
tions of Various Types of Disabilities by Employers," Psychologi- 
ca l Aspects of Disability . Vol. 18, 1971; C. A. Williams, "Is Hir- 
ing the Handicapped Good Business?" Journal of Rehabilitation . 
Vol. 38, 1972; James A. Colbert, Richard A. Kalisk, and Potter 
Chang, "Two Psychological Portals for Entry for Disadvantaged 
Groups," Rehabilitation Literature . July 1973; H.E. Yuker, 
"Attitudes of the General Public Toward Handicapped Individuals," 
in J.K. Weston (ed.) The White House Conference on Handicapped 
Individuals . Vol. 1, Washington, DC: U.S. Government Printing 
Office, 1977; John Schroedel and Richard Jacobson, Employer Atti- 
tudes Towards Hiring Persons with Disabilities . Alberston, New 
York: Human Resources Center, 1978. 

4. See, for example, Hill and Wehman, 1979; Kochany and Keller, 1982. 

5. Various official protections do exist, however, against an 
immediate loss of eligibility. 
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6. A similar situation is faced in other transfer payment programs, 
such as Aid to Families with Dependent Children, under which bene- 
fits are reduced as income levels rise. The financial return from 
working in some jobs may thus be lower than the benefits received 
when a person is not working (or is working less), thereby 
creating a disincentive for employment. 

7. Reflecting the complexity and confusion that sometimes surround 
these programs, MDRC discovered in informal discussions with staff 
members at several Social Security district offices early in the 
STETS demonstration that some uncertainty existed among the staff 
themselves about changes in SSI work incentive provisions that 
were incorporated into the 1980 Social Security Amendments. 

8. In 1979, national CETA regulations were amended in an effort to 
help handicapped individuals who were not members of economically 
disadvantaged families gain bitter access to CETA-funded employ- 
ment programs. The following change was enacted: "Handicapped 
persons will not be required to meet an income level to be consi- 
dered economically disadvantaged, provided that their handicaps 
present substantial barriers to employment." This st .idard is con- 
tinued under JTPA through a provision which allows Service 
Delivery Areas to enroll up to 10 percent of their participants 
from non-economical ly disadvantaged groups that face substantial 
employment barriers (Smith, 1983). 

9. For more information on these programs, see Moss, 1980; Wehman, 
1981; Schneider, Rusch, Henderson and Geske, 1982; Hill and 
Wehman, 1983. 

10. Some recent partial exceptions to this lack of attention to opera- 
tional and cost issues are studies of the several university-based 
employment programs focusing on more severely retarded persons. 
These programs have been designed and carefully monitored by 
university researchers. (See, for example, Hill and Wehman, 1983; 
Rusch; 1982, and Moss, 1980.) 

11. For a full report on the National Supported Work Demonstration, 
see MDRC, 1980. 

12. In addition to Nan Waterman, Richard Nathan and Gilbert Steiner of 
MDRC's Board, *the Advisory Committee included Edward Zigler, 
Sterling Professor of Psychology, Yale University; Phillip Roos, 
former Executive Director of Association f Retarded Citizens of 
the United States; Howard Rosen, private m* .ower consultant; Jay 
Rochlin, American Telephone and Telegraph company; and Lawrence 
Glantz, Human Resources Development Institute of the A.F.L.-C.I.O. 

13. MDRC recognized that an IQ score is an imperfect and controversial 
measure of functional ability. Therefore, the sites were also 
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allowed to consider applicants eligible for STETS if they were 
judged to be so by referral agencies using other criteria of 
retardation, provided that the evidence was verifiable. 



The original research design for the impact analysis called for a 
sample of 1,000 STETS applicants to be randomly assigned to the 
experimental and control groups. Follow-up interviewing was 
intended to continue for three years after random assignment. 
These plans had to be altered in response to funding constraints. 



CHAPTER TWO 



Compared to recruitment sources at other sites, DDD, the agency 
from which Tucson took many of its enrollees, served a higher 
proportion of the moderately retarded, who were more likely than 
higher-functioning individuals to be receiving SSI. Moreover, DDD 
counselors were likely to inform their clients if they were 
eligible for SSI, and this policy, too, may have raised the level 
of SSI recipients in Tucson's recruitment pool. 



CHAPTER III 



Of course, to some extent, these behaviors may reflect 
deficiencies in work habits as well as skills. 

Other studies have also identified attendance and tardiness 
problems, slow performance, inability to change routines, poor 
social skills, and "maladjustive behavior" as common reasons for 
job loss among mentally retarded persons. See, for example, Hill 
and Wehman, 1979, and Kochany and Keller, 1981. 

The division of responsibilities among staff members and the 
titles attached to similar staff positions varied across the 
sites. Therefore, for ease of presentation, this report will 
often use generic titles that reflect the major functions 
performed by staff members (e.g., "job developer," "counselor"). 

All names throughout this report have been changed to protect the 
identities of the people observed and interviewed. 

In the first eight months, a different agency provided these 
services, but a contract dispute prompted a shift to The Handlers, 
Inc. 

In addition, the St. Paul site hired as a consultant a vocational 
counselor, who had extensive experience in working with the 
mentally retarded, to assist in training participants with whom 
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the program had special difficulties. It also drew on the 
assistance of regular MDI job trainers. 

7. See Gold, 1973. 

8. These tests included WRAT, WREST and COATES. 

9. MIS data reveal that approximately one-third of the enrollees in 
St. Paul waited at least a month to begin Phase 1 activities. 

10. Through piece rates, Phase 1 participant-" in St. Paul tended to 
earn between $1.00 and $2.00 per hour. Few earned 100 percent of 
the minimum wage. 

11. This is apparent from Table 3.1, which shows the distribution of 
Phase 1 and 2 jobs according to their DOT cod^s. As an 
illustration, 56.3 percent of Phase 1 participants in Tucson were 
assigned benchwork jobs, while only 14.7 percent of Phase 2 
participants were placed in such jobs. 



CHAPTER IV 



1. Participants were considered to be "inactive" if they received no 
program services, in addition to not having a paid job. They were 
placed in a "hold" status if they received counseling or other 
support services for at least one hour per week. 

2. See Table A. 6 for data on the size of firms and agencies that 
accepted a STETS participant. 

3. It is interesting to note that an MDRC study of employers involved 
with an employment program for economically deprived youths (the 
Youth Incentive Entitlement Pilot Projects) found that 67 percent 
of employers cited "a chance to do something for disadvantaged 
youths" as one reason for bringing on a participant from that 
program, and 48 percent cited this as the most important reason 
(Ball, et al., 1981:37). 

4. Employers in public sector and nonprofit firms were excluded from 
the analysis of responses to this item because in the New York and 
Los Angeles tfites, numerous Phase 1 jobs extended into Phase 2. 
In such circumstances, Phase 2 participants would not have come to 
the employers with previous STETS training, since they began 
working for the organizations in Phase 1. 
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CHAPTER V 



1. Sec Appendix C for a more detailed discussion of the methodology 
for the employer survey* 

2. However, a comparison of Phase 2 participants who had been 
assigned to a supervisor for less than one month (but more than 
two weeks) to those who had been assigned for one month or more 
revealed no major differences in terms of supervisor evaluations 
of performance . 

3. For this analysis, the categories of these variables were defined 
as follows: age (18-20, 21-24), race (white, non-white), level of 
retardation (moderate, mild, borderline), secondary handicap (yes, 
no), high school completion (yes, no), prior work experience (yes, 
no), receipt of SSI and SSDI (yes, no). 

4. See, for example, Ingalls, 1978. 

5. On the basis of Fisher's Exact Test (two-tail), p«.0743 for race 
by productivity, p«.0484 for race by quality, and p«.0001 for race 
by training time, 

6. An ordinary least squares regression was also performed in an 
effort to examine the independent influence of each background 
variable on the three performance variables. done of the 
background factors was found to have significant influence, with 
the exception of receipt of SSI or SSDI on productivity (p«.0701). 
However, because each of the dependent variables is dichotomous 
with an uneven clustering of values across its two categories, the 
results- from this analysis may be biased and should therefore be 
interpreted cautiously. 

The only other relationship to show statistical significance — 
although marginal — was between training time and possession of a 
high school certificate or diploma at enrollment (p-.0961). Those 
with a certificate or diploma were more likely to require more 
training time. The reasons for this are unclear. 

7. Chapter 6 presents some preliminary data from the impact analysis 
on experimental-control differences at six months after random 
assignment. However, since many experimental group members were 
still in the program at that point, the findings do not reflect 
post-program outcomes. 

8. Because some participants changed jobs during Phase 3, only their 
first jobs during that phase are coasidered here. 

9. Additional information on fringe benefits is available from the 
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employer survey. Although it focused only on Phase 2 employers, 
many participants w*re already on the employers' payrolls, 
indicating that the chance* were good that their positions would 
evolve into Phase 3 placements. The benefits provided by their 
jobs are thus suggestive of what many participants could expect to 
receive as Phase 3 employees. As shown in Table A. 5, the vast 
majority of participants in the employer survey sample would be 
eligible for workmen's compensation, and half would receive health 
insurance. Over 63 percent would receive paid vacation days, but 
only one-quarter would get pensions with contributions made by the 
employer. 

10. For example, MIS data show that among the 83 percent of Phase 3 
participants who had at least one contact with the program during 
their first month after transition, 90 percent were employed at 
the same organization and 5 percent were employed at new 
organizations. The employment status of the 17 percent of 
participant who were not contacted during their first month in 
Phase 3 is not known. Among the 61 percent of Phase 3 
participants contacted after six months in that status, 84 percent 
were still employed, in either the same or a new organization. 
Again, the employment status of those not contacted at this point 
is not known. Because not all participants were contacted, these 
data on job retention must be interpreted cautiously. 



CHAPTER 6 



1. The research sample is drawn only from those program applicants 
who were randomly assigned to either experimental or control 
statu*. Therefore, the full sample includes all control group 
meir'^s, but only 88.4 percent of the 284 enrollees in STETS. 
Furthermore, follow-up interviews were attempted only for sample 
members who completed a baseline interview. 

2. The completion rate for the six-month survey, as with the 
baseline, was approximately 95 percent, resulting in a potential 
six-month sample of 293 individuals. However, due to the timing 
of the analysis for this report, the sample discussed here 
includes only 267 individuals. . 

3. See A. Bloomenthal et &1. , 1982, for complete details of the pilot 
study • 

4. Tables 6.2 and 6.3 also indicate that most of the differences 
calculated between the mean levels of labor market activities for 
the control and experimental groups are statistically signifi- 
cantly at least at the 10 percent level. This reinforces the view 
that the opportunities provided by STETS led to increased partici- 
pation in labor-market activities associated with training and to 
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decreased participation in substitute activities. Whils this 
statistical evidence is useful, it it important to reemphasize 
that the results may change in the final impact analysis, which 
will statistically adjust for baseline difference! between the 
experimental and control groups. 



CHAPTER 7 



For each site, the dates selected to represent the steady-state 
period were based on the five consecutive months during which the 
program served a relatively high number of active participants 
compared to the number served in the other months of the 
demonstration. 

Five-month intervals were also used for calculating the sites' 
steady-state service project revenue and employers' direct 
payments to participants. However, the start of this period in 
each site was defined to occur three months after the start of the 
steady-state period. This was to reflect the delays experienced 
by the sites in receiving revenues. 

Enrolled time refers to the time from the date of random 
assignment to the date of exit from the program. It should be 
recalled from Chapter III that participants were not always 
"active" (i.e., directly receiving program services) during this 
time. However, it is also important to note that staff efforts 
were often devoted to participants while they were inactive. This 
would include time spent developing jobs, maintaining clients' MIS 
and other program records, and checking periodically on their 
status and well-being. 

Research-related activities included the additional effort to 
recruit, interview and randomly assign twice as many applicants as 
were enrolled in the program (since half were assigned to the 
control group); a portion of the time devoted to completing MIS 
and other paperwork related to the research; and the time spent by 
staff members in research-related interviews and meetings with 
MDRC researchers and field representatives. Project directors 
were asked to estimate, for each member of their staff, the propor- 
tion of time that was devoted to these activities. The cost of 
these activities was then derived by multiplying the time 
estimates by the labor costs for the staff members involved. The 
average cost for research-related activities was thereby deter- 
mined to be about 5 percent of the sites' operating expenses. 
Each site's total reported costs were thus reduced by 5 percent 
before adjusting for service project revenue. 

The following formulas were used in calculating net unit costs, 
after reducing total costs by 5 percent for research-related 
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expense i. 



Net Cost Per Service Year : Total cost for the steady-state period 
minui service project revenue, divided by the turn of the number of 
participants enrolled during each of these months, multiplied by 
12 months. 

Net Cost Per Participant : Total cost for the steady-state period 
minus service project revenue, divided by the sum of the number of 
participants enrolled during these months, multiplied by the 
average length of stay (enrolled time) in the program. 

It is important to note that the amount of each site's service 
project revenue and direct employer payments to participants was 
somewhat dependent upon the relative proportion of participants in 
Phase 1 versus Phase 2, since such revenue was generally higher 
for Phase 2 jobs, many of which were unsubsidized. Consequently, 
these estimates are sensitive to the date defining the end of the 
steady-state period in each site, because the ratio of Phase 2 
participants was higher later in the demonstration. For the 
demonstration as a whole, almost two-thirds of all active 
participants in STETS were in Phase 1 during the steady-state 
period. This is different from what might be expected during a 
more extended normal operating period, during which more 
participants are likely to be in Phase 2 than in Phase 1, since 
the average length of stay is likely to be longer in the second 
phase of the program. Because the reverse was true in this 
analysis, the actual ongoing cost of operating STETS ««y be lower 
than the estimates reported here. 

It is also apparent from Table 7.1 that Cincinnati's net operating 
cost was somewhat lower than New York's because the combination of 
service project revenue and direct employer payments was higher; 
without these offsetting monies, Cincinnati vould have had a 
higher operating cost than New York. Among all the sites, New 
York had the lowest amount of service project revenue and employer 
payments. This is because, in contrast to other sites, New fork's 
Phase 2 participants were generally in subsidized training 
positions; an unsubsidized position marked the beginning of Phase 
3 at that site. In some cases, however, the New York program was 
able to obtain reimbursements for participants' wages through its 
"fee for service 11 arrangement, which generally lasted from 2 to 12 
weeks • 

If it is assumed that a dollar of benefit or cost to one person 
equals a dollar of benefit or cost to any other person, the 
participant and nonparticipant perspectives can be added together 
to obtain the social perspective. This statement holds because 
the participant and nonparticipant perspectives are defined to be 
mutually exclusive and exhaustive. Thus, when the findings from 
these perspectives are added together, only benefits or costs that 
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accrue to one group with no offsetting cost or benefit to the 
other group (i.e., those involving the use, rather than the 
redistribution, of resources) will remain. 

Thus, the value of output should not be less than what was 
actually paid. Of course, this argument is weakened if altruism 
prompts employers to overpay because of their desire to support 
the STETS program and its participants. In either event, direct 
estimates (discussed next in the text) offer a more accurate 
estimate of the resource value of participant output. 

The studies were completed between September and December 1982. 
The two-week reference periods, defined individually for each 
site, corresponded to the two weeks prior to the site visit. 
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TABLE A.l 



PERCENTAGE DISTRIBUTION AND AVERAGE LENGTH OF ACTIVE TIME 
IN 5TETS, BY PHASE AND PROGRAM SITE 



Length of Active Time 


Los 
Angeles 


New 
York 


Cin- 
cinnati 


St. Paul 


Tucson 


All 
Sites 


Phase 1* 














1 Month or Less 
2-3 Months 
4-5 Months 
6—7 Months 
8-9 Months 
10-12 Months 
More Than 12 Months 
Total 


6.9 

20 7 
55.2 

5.2 
3.4 
0.0 
100.0 


6.2 

90 9 

49.2 
13 .8 
0.0 
0.0 
1.5 
100 .0 


3.3 

d. J , J 

25.0 
35.0 
5.0 
5.0 
3.3 
100 .0 


11.4 
20 .5 
20 .5 
15.9 
6.8 
13.6 
11.4 
100 .0 


2.0 
9.8 
15.7 
23.5 
21.6 
17.6 
9.8 
100.0 


5.8 
21 .2 
34.5 
19.4 
7.2 
7.2 
4.7 
100 .0 


Average Duration (Months) 


3.9 


3.5 


4.9 


5.8 


7.5 


5.0 


Number of Enrol lees 


58 


65 


60 


44 


51 


278 


Phase 2 














1 Month or Less 
2-3 Months 
4-5 Months 
6-7 Months 
8-9 Months 
10 - 12 Months 
More Than 12 Months 
Total 


2.8 
13 .9 
25.0 

ft 1 

16.7 
33 .3 
0.0 
100.0 


9.6 
23 1 
19.2 

1 5 A 

15.4 
13.5 
3.8 
100 .0 


5.0 

37.5 

10.0 
2.5 
0.0 
100.0 


4.5 

Art O 

22.7 
4.5 

27 .3 
0.0 
0.0 
100 .0 


5.6 

11 1 
11.1 

25.0 
19.4 
22.2 
8.3 
8,3 
100.0 


5.9 
18.8 
25.8 
17 .2 
17.2 
12.4 
2.7 
100 .0 


Average Duration (Months) 


6.4 


5.5 


4.6 


4.1 


6.4 


5.5 


Number of Enrol lees 


36 


52 


40 


22 


36 


186 c 


Phases 1 and 2 (Combined)* 1 














1 Month or Less 
2-3 Months 
4-5 Months 
6-7 Months 
8-9 Months 
10-12 Months 
More Than 12 Months 
Total 


6.9 
12.1 
13.8 
15.5 
12.1 
13.8 
25.9 
100.0 


3.1 
7.7 
13.8 
15.4 
20.0 
16.9 
23.1 
100 .0 


3.3 
11.7 

6.7 
21.7 
10.0 
33.3 
13.3 
100.0 


11.4 
11 .4 
13.6 
15.9 
11.4 
13.6 
22.7 
100.0 


2.0 
3.9 
9.8 
17.6 
2.0 
17.6 
47.1 
100 .0 


5.0 
9.4 
11.5 
17.3 
11.5 
19.4 
25.9 
100 .0 


Average Duration (Months) 


7.9 


8.3 


8.0 


7.8 


11.5 


8.6 


Number of Enrol lees 


58 


65 


60 


44 


51 


278 



(continued) 



TABLE A.l (continued) 

SOURCE: MDRC tabulations of Monthly Status Change Forms m the STETS Intormati 
System. 

NOTES; Percentage distributions may not add exactly to 100.0 because of 
rounding. 

B 

Includes Phase 1 active time for enrollees who entered Phase 1. 

includes Phase 2 active time for enrollees who entered Phase 2. 
c 

Excludes one enrol lee in Los Angeles for whom data are missing. 

includes Phase 1 and Phase 2 active time for enrollees who were ever 
active in STETS. 
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TABLE A. 2 

SPEARMAN RANK-ORDER CORRELATION COEFFICIENTS FOR SELECTED DIMENSIONS 
OF VORK PERFORMANCE BY PHASE 2 PARTICIPANTS 3 



Dimension of 
Performance 



Attendance 

Getting to Work On Time 

Motivation 

Aaount 0 f Work Done 
(Productivity) 

Quality of Work Done 

Getting Along With Co-Worker* 



Getting to Work , . 

Attendance On Tim* u « , Quality of Getting Along 

Attendance On Time Motivation Productivity Work Done With Co-Workera 



1 .0000 



.3957 
1 .0000 



.3396 


.3504 


.0157 


.0840 


.0228 


.1099 


.1897 


.0342 


1 .0000 


.4567 


.2640 


.3251 




1 .0000 


.5037 


.1809 



1 .0000 



.0938 
1 .0000 



SOURCE: 



MDRC t.bul.tion. of d.t. froo .upervi.or repon.e. to the STETS Employer Survey In.trun.ent. 
1982 Z%H ^TsZluli^ !: i l'5: ed t0 Cmpl0y "' ° f Ph " e 2 P«ti=i P .nt. .t .11 five STETS .ite. between Dece.ber 



Reaponaea of "Don't Know" were omitted f 
any correlation coefficient. 



rom the calculations; *t moit, these included 3 obie 



rvatiom for 



.u P ervi.or ev.lu.tion of he S?£s w&wtZ^tT™*" 11 *^ ^ dich «~«" "tegorie. b..ed upon 
With Co-Vorker.,» . v.lue of »PwSEJ!T|fTS^M,? °l V*™ in ,iniUr j° b< - F ° r "Getting Along 

v.lue „f »0" ... igned for LtliTC Too W nS" oor C W l » ed " V <7 Wel >" Well"; 8 , 

•signed if . p.rticip.nt received I r.ting of "A. Well A. » "ToLZL, I „ » v * ^ 1 l * ble, • ■ "1" of "1" „.. 

«-ployee. in .i„U. r job. for .bout the ,J e Lount » Ml. ? "c^tS" Th * n ' ° r Much Better '•«" ° th « 
Wor.e" «nd "Much Wor.e." ° f tl ° e ' * V * lue of 0 w " -"igned for r.ting. of "Somewhat 



BEST COPY AVAILABLE 
194 
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TABLE A. 3 



REASONS CITED BY SUPERVISORS FOR .OWER PRODUCTIVITY AND LOWER QUALITY 
OF WORK AMONG PHASE 2 PARTICIPANTS COMPARED TO 
OTHER NEW WORKERS IN SIMILAR JOBS 



Reason 



Productivity 

Poor Motivation or Concentration 
Difficulty in Understanding, Remembering 

or Doing Tasks 
Performed Tasks More Slowly 
Physical or Other Problems 

Number of Participants 

Quality 

Lower Motivation or Bad Attitude 
Difficulty Understanding or Remembering Tasks 
Work Not Done Correctly or Carefully j 



Number of Participants 



Percent of 
Participants 



24.1 

37.9 
51.7 
13.8 

29 



19.1 
33 .3 
57.1 

21 



SOURCE: MDRC tabulations of data s from supervisor responses to the 
STETS Employer Survey la^trument. 

NOTES: The survey was administered to employers of Phase 2 
participants at all five STETS sites between December 1982 and April 1983. 

More than one reason was cited for some participants. 

This table includes those participants who received a rating 
of "Somewhat Worse 11 or "Much Worse" for "Amount of Work Done" 
(Productivity) and "Quality of Work Done." 
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TABLE A. A 



PERCENTAGE DISTRIBUTION OF SUPERVISOR REACTIONS 
TO THE AMOUNT OF TIME COUNSELORS SPENT WITH PHASE 2 PARTICIPANTS 
AT THE WORKSITE, BY AMOUNT OF TIME SPENT PER WEEK 



Supervisor Reaction 8 


Less than 1 Hour 


1 Hour or More 


Positive or Neutral 


91 .4 


79,2 


Negative 


8.6 


20.8 


Total 


100.0 


100 .0 


Number of Participants^ 


58 


24 



SOURCE: MDRC tabulations of data from supervisor responses 
to the STETS Employer Survey Instrument. 

NOTES: The survey was administered to employers of Phase 
2 participants at all five STETS sites between December 1982 and 
April 1983. 

A supervisor s reaction was coded as "positive or 
neutral" when he or she wished that the counselor had spent the 
same or more time than was actually spent with the participant, 
or said that it did not matter. A supervisor's reaction was 
coded as "negative" if he or she wished that the counselor spent 
less time than was actually spent with the participant. 

b Excludes 18 STETS participants whose supervisors, 
when asked their reaction, answered "Don't Know" or did not 
respond . 
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TABLE A. 5 



PERCENT OF PHASE 2 PARTICIPANTS ON THE EMPLOYER'S 
PAYROLL WHO RECEIVED SELECTED FRINGE BENEFITS 



Fringe Benefit 


Percent or Participants* 


Workmen's Conversation 


88.5 


Paid Vacation Days 


63.5 


Health Insurance 


50 .0 


Paid Holidays 


48.1 


Paid Sick Days 


39.1 


Pension 


25.0 


Number of Participants 


52 



SOURCE: MDRC tabulations of data from 
employer responses to the STETS Employer Survey 
Instrument • 

.NOTES: The survey was administered to 
employers of Phase 2 participants at all five 
STETS sites between December 1982 and April 
1983. 

Some participants received more 
thap one fringe benefit from employers. 
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TABLE A. 6 



PERCENTAGE DISTRIBUTION OF SIZE OF FIRMS 
OR AGENCIES THAT ACCEPTED A PHASE 2 PARTICIPANT 



Number of Workers* 


Percent of Finns/ 
Agencies 


Fewer Than 25 


3.0 


25-49 


12.1 


50-99 


8.1 


100-249 


28.3 


250-499 


20.2 


500-999 


18.2 


1000 or more 


9.1 


Don't Know 


1.0 


Total 


100.0 


Number of 





Firms/Agencies f 99 



SOURCE: MDRC tabulations of data from 
employer responses to the STETS Employer Survey 
Instrument . 

NOTES: The 6urvey was administered to 
employers of Phase 2 participants at all five 
STETS sites between December 1982 and April 
1983. 

a 

The number of workers was 
calculated by adding the number of full-time 
workers and one-half of the number of part-time 
workers at the firm/agency. If the number of 
part-time workers was unknown, the average 
number of part-time workers among organizations 
whose number of part-time workers was known was 
used as a proxy. 
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APPENDIX B 

TECHNICAL ASPECTS OF THE EMPLOYER SURVEY 
AND JOB DEVELOPMENT CONTACT STUDY 



-177- 

199 



APPENDIX B 

TECHNICAL ASPECTS OF THE EMPLOYER SURVEY 
AND JOB DEVELOPMENT CONTACT STUDY 

I. Employer Survey 

The STETS employer survey, conducted between December 1982 and April 
1983 in all five sites, was administered to employers who at the time of 
the interview were employing or had employed Phase 2 trainees. The 
interview was divided into two sections. The first part, which was 
administered to the person at the firm or agency who made the decision to 
bring on a STETS participant, inquired about the reasons for that decision. 
The second section was addressed to the person who was serving or had 
served as the trainee's direct supervisor (sometimes, but not always, 
synonymous with the person who made the hiring decision). Supervisors were 
asked to evaluate the performance of their trainees and to describe their 
contact with and reactions to the STETS counselors who visited the 
worksite. 

All worksites where a Phase 2 participant had worked for at least two 
weeks were contacted. The response rate was high: There were answers from 
93 percent of the x23 worksites contacted, although for 11 percent of the 
responses only one of the two ra^in parts of the interview was completed. 

Some supervisors had worked with more than one Phase 2 participant. 
To minimize the burden on those respondents, they were asked to evaluate 
only up to two participants. In the few cases where more than two 
participants had worked for a given supervisor, the names of two 
participants to be discussed were selected randomly, except that priority 
was given to participants currently at the worksite. Also, in cases where 
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„ participants had been assigned to more than one worksite during Phase 2, 
only the most recent of their supervisors was asked for an evaluation. 
y Because not all Phase 2 participants were evaluated, it is necessary 

to determine whether the survey sample was representative of the larger 
group. To answer this question, MIS data on selected background charac- 
teristics were compared for participants included and not included in the 
survey. The differences between the samples were not statistically 
significant for age, sex, le^el of retardation, presence of secondary 
handicap, ethnicity, and possession of a high school diploma or equivalency 
Certificate. Those included L the survey were somewhat less likely to 
have participated in an employment or training program prior to STETS, and 
they were also somewhat less likely to have been receiving SSI or SSDI at 
enrollment. In general, though, the analysis suggested that individuals 
included in the survey were representative of all Phase 2 participants. 

Survey data were collected primarily through telephone interviews by 
MDRC researchers. However, for 19 percent of the worksites in the sample, 
the interview was conducted in-person by a researcher from Mathematica 
Policy Research (MPR), as part of the value of output analysis for the 
benefit-cost evaluation. (See Chapter VII.) This eliminated the need to 
impose two separate interviews on employers and supervisors. To assess the 
potential bias of using these two approaches, the firms and agencies 
interviewed by MDRC and MPR were compared in terms of participants" 
performance evaluations and ether selected variables, such as size of 
staff, sector and types of fringe benefits provided. In general, the 
differences between the two groups of interviews were minor. 
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II. Job Development Contact Study ^ 

The job development contact study was conducted in all sites from 
September 1, 1982 through April 30, 1983. However, due to problems in 
implementing the study, data from the Los Angeles site were not included in 
the analysis. Using special forms prepared by MDRC, job developers kept a 
record of the contacts they made with employers during the study period. 
The study made a distinction between "preliminary" and "substantive" 
contacts. Preliminary contacts were documented for all employers who were 
reached during the study period and had not previously been contacted by 
the program. A contact was considered substantive if the job developer was 
able to talk with the employer long enough to explain the key elements of 
STETS . 

For preliminary contacts, the job developers recorded, among other 
information, whether or not the employer gave an immediate refusal or was 
interested in learning more about the program. They also noted any 
instance in which a preliminary contact led to a substantive one. For 
substantive contacts, they recorded the sector and type of business, the 
source through which the organization had been identified and the 
employer's response. Any follow-up contacts with employers who indicated 
that they were interested in the program but had no available openings were 
documented for up to 90 days after the first substantive contact. 
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APPENDIX C 

DATA COLLECTIO N AND ANALYTICAL METHODS FOR THE IMPACT ANALYSIS 

I. .Interviewing the Mentally Retarded 

Because of the potential difficulty of interviewing mentally retarded 
persons, a pilot study was undertaken by Mathematica Policy Research to 
assess the completeness and quality of data that such an effort would 
yield. The study was conducted with 103 early STETS enrollees in 
Cincinnati, New York and Tucson. For each sample member, responses to the 
in-person interviews were compared with information obtained from records 
of the STETS agency and with responses to in-person interviews with proxy 
respondents (parents, guardians or counselors), using identical questions. 
Completion rates were high for both orimary and proxy respondents (95 and 
99 percent, respectively). 

Findings indicated that it was feasible to collect data based on 
self-reported interviews with the STETS sample members. The incidence of 
missing or incomplete data in the interviews with primary respondents was 
at least as low as in the interviews with proxies; and there was some 
evidence that the proxy interview data were as susceptible to error as data 
from the primary interviews, or even more so. Results also indicated, 
however, that proxy reports could generally be used to replace or 
supplement missing information on specific items in the participant 
interviews. Furthermore, there was a great deal of consistency between 
data from the two sets of interviews. 

The study concluded that it might be necessary to use proxy 
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interviews for approximately 30 percent of the sample for the baseline and 
follow-up surveys; however, for the six-month follow-up interview, they 
were necessary in only 19 percent of the cases. Furthermore, when proxy 
interviews were necessary, they were almost always available (in over 96 
percent of the cases in which they were called for). 

In the analysis for this report, the proxy interview served as the 
basic source for six-month data for 6 percent of the sample members. For 
an additional 11 percent of the sample members, proxy-interview data 
substituted for primary interview data for selected items. 

In the interviews, abstract concepts were, whenever possible, broken 
down into discrete components. For example, rather than being asked to 
recall the number of hours they worked per day, sample members were asked 
what time they started work, when they stopped and how much time they had 
for a lunch break. Similarly, independence in financial management was 
measured by a series of questions about shopping, bill-paying and banking. 

Also to improve accuracy, redundancy was explicitly built into the 
employment and training module! ;o enable respondents to report on these 
activities through responses to any one of a series of questions. 
Furthermore, in each of the four surveys, respondents were asked only about 
current and very recent activities to avoid the confusion that mighr result 
from a lengthy recall period* This "point-in-time" approach, which 
improves the quality of the data at the expense of completeness, represents 
a necessary compromise for this population. 
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II, Activity Measures in the Six-Month Interview 

In the anaLysis of the activities of treatment and control group 
members at six months after random assignment, when a considerable share of 
treatment group members were still participating in STETS, three specific 
outcomes were studied: (1) labor-market activity; (2) financial management 
skills; and (3) the extent to which sample members relied on Supplemental 
Security Income (SSI) and Social Security Disability Insurance (SSDI) 
benefits and the amount of such benefits. The first two of these measures 
are discussed in more detail below. 

Labor Market Activity . To define the most important labor market acti- 
vity of sample members, the researchers created a hierarchy of activities 
arranged from "highest 11 to "lowest. 11 Thfy were as follows: regular (or 
non-training-related) paid jobs (highest), training jobs, participation in 
sheltered workshops/activity centers, unpaid jobs, training without a job 
attached, attendance in school, and inactive (i.e., none of the preceding). 

Respondents were placed in the regular category only when they indi- 
cated that their jobs were not "part of a training program about work." The 
training jobs category included jobs reported to be (1) "part of a training 
program about work" or "work-study" jobs, or (2) part of a school 
curriculum including paid work. The sheltered workshop category included 
all paid jobs at workshops and activity centers, whether or not they were 
reported to be part of a training program. The unpaid jobs category cover- 
ed a wide range of activities, including such unpaid work-study positions 
as being a cafeteria aide or sports-team manager. While a review of case 
records shows that most such jobs entailed specific hours and 
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responsibilities, many had a large social or recreational component. 

Each individual was assigned to the highest category for which parti- 
cipation was reported, even if he or she might have spent more time in an 
activity assigned to a lower category. A STETS job was assigned to either 
the training -job or she ltered workshop category, depending on the nature of 
the placement. If a sample member had held more than one paid job, the 
researchers focused on the one with the most regularity and longest hours. 
When interview data were ambiguous, they were refined with information on 
the services provided by the agency that the respondent reported had 
provided the job or training. 

Financial Manageme nt Skills. A composite measure of financial manage- 
ment skills was used. It was constructed from questions on three aspects 
of money-handling and financial activities: banking, shopping and pur- 
chasing, and bill-paying. The sample member was defined as having such 
skills if he or she was involved in at least two of the three activities 
and was able to conduct each without assistance. Given the problems facing 
mentally retarded young adults and the fact that they would have to have 
not only the ability but also the opportunity to perform such activities, 
this may be a strict standard. 



III. Analytical Method 

As noted earlier, the impact analysis for the STETS evaluation used 
point-in-time data on sample members" activities and experiences. Such 
data do not enable the analyst to estimate the length of employment or 
service use during the first six months post-baseline. In subsequent 
analyses, many of these patterns of employment and service use will be 
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estimated "through linear extrapolation between points of observation. At 
present, however, the analysis is confined to evaluating sample members' 
activities at the first point-in-time observation. 
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